Jun 03 15:40:52.514474 ip-10-0-129-129 systemd[1]: Starting Kubernetes Kubelet... Jun 03 15:40:52.870356 ip-10-0-129-129 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 15:40:52.870356 ip-10-0-129-129 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 03 15:40:52.870356 ip-10-0-129-129 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 15:40:52.870356 ip-10-0-129-129 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 03 15:40:52.870356 ip-10-0-129-129 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 15:40:52.872966 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.872868 2578 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 03 15:40:52.876234 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876218 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 15:40:52.876234 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876234 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876238 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876241 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876244 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876246 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876249 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876253 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876256 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876260 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876265 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876269 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876272 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876275 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876278 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876282 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876285 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876293 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876296 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876299 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876302 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 15:40:52.876304 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876304 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876307 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876311 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876314 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876317 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876319 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876322 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876324 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876327 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876330 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876332 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876335 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876337 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876340 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876343 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876346 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876348 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876351 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876353 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876356 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 15:40:52.876857 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876359 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876361 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876364 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876366 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876369 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876371 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876374 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876377 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876379 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876382 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876387 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876391 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876395 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876397 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876400 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876403 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876405 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876408 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876410 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 15:40:52.877368 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876413 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876416 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876418 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876420 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876423 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876425 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876428 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876430 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876433 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876435 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876438 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876440 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876443 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876445 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876447 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876450 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876452 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876455 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876457 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 15:40:52.877832 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876460 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876462 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876465 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876467 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876470 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876473 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876476 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876856 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876862 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876865 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876868 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876871 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876874 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876876 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876879 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876882 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876884 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876887 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876889 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876892 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 15:40:52.878349 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876895 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876897 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876900 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876902 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876905 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876907 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876910 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876913 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876915 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876918 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876920 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876923 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876925 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876928 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876930 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876933 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876936 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876939 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876942 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876944 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 15:40:52.878823 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876949 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876953 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876957 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876960 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876963 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876966 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876969 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876972 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876975 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876977 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876980 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876982 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876985 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876988 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876990 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876993 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876995 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.876997 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877000 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 15:40:52.879335 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877003 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877005 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877008 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877011 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877013 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877015 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877018 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877021 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877023 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877026 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877029 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877032 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877035 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877037 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877040 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877042 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877045 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877047 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877050 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877052 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 15:40:52.879809 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877059 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877062 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877064 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877066 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877069 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877071 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877074 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877077 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877079 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877082 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877085 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877087 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877090 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.877092 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877634 2578 flags.go:64] FLAG: --address="0.0.0.0" Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877643 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877650 2578 flags.go:64] FLAG: --anonymous-auth="true" Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877654 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877662 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877666 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877670 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 03 15:40:52.880339 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877675 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877679 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877682 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877686 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877689 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877692 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877695 2578 flags.go:64] FLAG: --cgroup-root="" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877698 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877701 2578 flags.go:64] FLAG: --client-ca-file="" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877704 2578 flags.go:64] FLAG: --cloud-config="" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877707 2578 flags.go:64] FLAG: --cloud-provider="external" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877710 2578 flags.go:64] FLAG: --cluster-dns="[]" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877714 2578 flags.go:64] FLAG: --cluster-domain="" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877717 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877720 2578 flags.go:64] FLAG: --config-dir="" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877723 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877729 2578 flags.go:64] FLAG: --container-log-max-files="5" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877733 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877736 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877740 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877743 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877746 2578 flags.go:64] FLAG: --contention-profiling="false" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877749 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877752 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877755 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 03 15:40:52.880851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877758 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877762 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877765 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877768 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877771 2578 flags.go:64] FLAG: --enable-load-reader="false" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877774 2578 flags.go:64] FLAG: --enable-server="true" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877777 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877782 2578 flags.go:64] FLAG: --event-burst="100" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877785 2578 flags.go:64] FLAG: --event-qps="50" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877788 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877791 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877795 2578 flags.go:64] FLAG: --eviction-hard="" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877799 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877801 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877804 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877808 2578 flags.go:64] FLAG: --eviction-soft="" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877810 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877813 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877816 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877819 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877822 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877825 2578 flags.go:64] FLAG: --fail-swap-on="true" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877828 2578 flags.go:64] FLAG: --feature-gates="" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877834 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877837 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 03 15:40:52.881467 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877840 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877844 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877849 2578 flags.go:64] FLAG: --healthz-port="10248" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877852 2578 flags.go:64] FLAG: --help="false" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877855 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-129-129.ec2.internal" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877858 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877861 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877864 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877867 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877870 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877873 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877876 2578 flags.go:64] FLAG: --image-service-endpoint="" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877879 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877882 2578 flags.go:64] FLAG: --kube-api-burst="100" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877885 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877888 2578 flags.go:64] FLAG: --kube-api-qps="50" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877891 2578 flags.go:64] FLAG: --kube-reserved="" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877893 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877896 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877900 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877902 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877905 2578 flags.go:64] FLAG: --lock-file="" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877908 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877911 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 03 15:40:52.882083 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877914 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877919 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877922 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877925 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877928 2578 flags.go:64] FLAG: --logging-format="text" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877931 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877936 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877939 2578 flags.go:64] FLAG: --manifest-url="" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877941 2578 flags.go:64] FLAG: --manifest-url-header="" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877946 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877949 2578 flags.go:64] FLAG: --max-open-files="1000000" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877953 2578 flags.go:64] FLAG: --max-pods="110" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877956 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877959 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877962 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877965 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877968 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877971 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877974 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877982 2578 flags.go:64] FLAG: --node-status-max-images="50" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877985 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877988 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877991 2578 flags.go:64] FLAG: --pod-cidr="" Jun 03 15:40:52.882683 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.877994 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878000 2578 flags.go:64] FLAG: --pod-manifest-path="" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878003 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878006 2578 flags.go:64] FLAG: --pods-per-core="0" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878009 2578 flags.go:64] FLAG: --port="10250" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878012 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878015 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0af6bd92104cafdd5" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878019 2578 flags.go:64] FLAG: --qos-reserved="" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878022 2578 flags.go:64] FLAG: --read-only-port="10255" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878025 2578 flags.go:64] FLAG: --register-node="true" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878028 2578 flags.go:64] FLAG: --register-schedulable="true" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878032 2578 flags.go:64] FLAG: --register-with-taints="" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878040 2578 flags.go:64] FLAG: --registry-burst="10" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878043 2578 flags.go:64] FLAG: --registry-qps="5" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878046 2578 flags.go:64] FLAG: --reserved-cpus="" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878050 2578 flags.go:64] FLAG: --reserved-memory="" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878054 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878057 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878060 2578 flags.go:64] FLAG: --rotate-certificates="false" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878063 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878066 2578 flags.go:64] FLAG: --runonce="false" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878069 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878072 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878075 2578 flags.go:64] FLAG: --seccomp-default="false" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878078 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878081 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 03 15:40:52.883285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878084 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878087 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878090 2578 flags.go:64] FLAG: --storage-driver-password="root" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878093 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878097 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878099 2578 flags.go:64] FLAG: --storage-driver-user="root" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878102 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878105 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878108 2578 flags.go:64] FLAG: --system-cgroups="" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878111 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878116 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878119 2578 flags.go:64] FLAG: --tls-cert-file="" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878122 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878126 2578 flags.go:64] FLAG: --tls-min-version="" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878129 2578 flags.go:64] FLAG: --tls-private-key-file="" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878132 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878138 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878141 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878144 2578 flags.go:64] FLAG: --v="2" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878149 2578 flags.go:64] FLAG: --version="false" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878153 2578 flags.go:64] FLAG: --vmodule="" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878159 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878162 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878286 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 15:40:52.884290 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878291 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878294 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878297 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878300 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878302 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878305 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878308 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878311 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878315 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878319 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878321 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878324 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878328 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878331 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878334 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878337 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878340 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878343 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878345 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 15:40:52.885061 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878348 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878351 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878354 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878357 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878360 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878364 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878366 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878369 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878372 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878374 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878379 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878381 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878384 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878387 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878390 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878392 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878395 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878398 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878400 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878403 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 15:40:52.885579 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878406 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878408 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878411 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878413 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878416 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878418 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878420 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878423 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878426 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878429 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878431 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878434 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878436 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878439 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878442 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878444 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878447 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878451 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878454 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 15:40:52.886072 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878456 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878459 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878461 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878469 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878471 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878474 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878477 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878479 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878482 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878484 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878487 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878489 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878492 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878495 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878497 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878500 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878502 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878505 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878508 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878510 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 15:40:52.886569 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878513 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 03 15:40:52.887076 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878515 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 15:40:52.887076 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878518 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 15:40:52.887076 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878520 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 15:40:52.887076 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878523 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 15:40:52.887076 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878525 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 15:40:52.887076 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.878528 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 15:40:52.887076 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.878533 2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 15:40:52.887076 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.887046 2578 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 03 15:40:52.887076 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.887064 2578 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887116 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887121 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887124 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887127 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887130 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887133 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887136 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887139 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887141 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887144 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887147 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887150 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887152 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887155 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887157 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887160 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887163 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887165 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887168 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 15:40:52.887364 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887170 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887173 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887176 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887179 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887181 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887185 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887188 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887190 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887193 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887208 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887212 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887217 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887220 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887223 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887227 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887230 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887233 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887236 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887239 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887241 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 15:40:52.887859 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887244 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887246 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887249 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887252 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887254 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887257 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887261 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887264 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887268 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887271 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887274 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887277 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887279 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887282 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887285 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887287 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887290 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887292 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887295 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 15:40:52.888371 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887297 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887300 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887302 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887305 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887307 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887310 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887313 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887316 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887319 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887322 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887325 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887328 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887331 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887334 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887337 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887340 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887342 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887345 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887348 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887350 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 15:40:52.888827 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887353 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887356 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887358 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887361 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887363 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887367 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887372 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887375 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.887381 2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887479 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887484 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887488 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887490 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887493 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887496 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 15:40:52.889341 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887499 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887502 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887504 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887508 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887513 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887516 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887519 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887522 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887525 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887527 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887530 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887532 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887535 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887538 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887540 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887543 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887546 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887549 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887551 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887554 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 15:40:52.889711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887556 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887559 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887561 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887563 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887566 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887568 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887571 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887573 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887575 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887578 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887581 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887583 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887586 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887589 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887591 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887594 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887597 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887599 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887602 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887605 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 15:40:52.890231 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887607 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887611 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887615 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887618 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887620 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887623 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887625 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887628 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887630 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887633 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887636 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887638 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887641 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887644 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887665 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887669 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887672 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887675 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887678 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 15:40:52.890778 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887681 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887684 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887686 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887689 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887692 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887694 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887697 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887700 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887703 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887706 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887709 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887711 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887714 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887716 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887719 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887722 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887724 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887727 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887730 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887732 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 15:40:52.891264 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:52.887735 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 15:40:52.891766 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.887739 2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 15:40:52.891766 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.888413 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jun 03 15:40:52.891766 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.891518 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 03 15:40:52.892293 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.892261 2578 server.go:1019] "Starting client certificate rotation" Jun 03 15:40:52.892380 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.892365 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 15:40:52.892413 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.892405 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 15:40:52.911239 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.911219 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 15:40:52.913078 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.913059 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 15:40:52.931138 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.931117 2578 log.go:25] "Validated CRI v1 runtime API" Jun 03 15:40:52.936245 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.936230 2578 log.go:25] "Validated CRI v1 image API" Jun 03 15:40:52.937398 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.937384 2578 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 03 15:40:52.940778 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.940756 2578 fs.go:135] Filesystem UUIDs: map[00142af9-17d9-4fc1-a434-b211b69c7d6f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a53c7d58-f0b0-42d9-9e7c-7df1f7fcbc2e:/dev/nvme0n1p3] Jun 03 15:40:52.940848 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.940778 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 03 15:40:52.942349 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.942321 2578 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 15:40:52.945242 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.945116 2578 manager.go:217] Machine: {Timestamp:2026-06-03 15:40:52.944602741 +0000 UTC m=+0.329669951 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099649 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2405af5a0465567d7cfa764ac5b968 SystemUUID:ec2405af-5a04-6556-7d7c-fa764ac5b968 BootID:16b2c232-5fb6-499c-ba70-7e0d311f64b6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d0:20:e1:0b:e3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d0:20:e1:0b:e3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:43:fd:fa:dd:4e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 03 15:40:52.945839 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.945828 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 03 15:40:52.945927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.945916 2578 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 03 15:40:52.947565 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.947538 2578 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 03 15:40:52.947754 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.947568 2578 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-129.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 03 15:40:52.947795 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.947764 2578 topology_manager.go:138] "Creating topology manager with none policy" Jun 03 15:40:52.947795 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.947773 2578 container_manager_linux.go:306] "Creating device plugin manager" Jun 03 15:40:52.947795 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.947781 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 15:40:52.948383 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.948372 2578 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 15:40:52.949598 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.949588 2578 state_mem.go:36] "Initialized new in-memory state store" Jun 03 15:40:52.949707 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.949699 2578 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 03 15:40:52.951583 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.951573 2578 kubelet.go:491] "Attempting to sync node with API server" Jun 03 15:40:52.951621 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.951586 2578 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 03 15:40:52.951621 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.951597 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 03 15:40:52.951621 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.951606 2578 kubelet.go:397] "Adding apiserver pod source" Jun 03 15:40:52.951621 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.951615 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 03 15:40:52.952661 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.952643 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 15:40:52.952718 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.952671 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 15:40:52.955525 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.955508 2578 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 03 15:40:52.957237 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.957221 2578 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 03 15:40:52.959321 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.959306 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 03 15:40:52.959403 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.959330 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 03 15:40:52.959403 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.959340 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 03 15:40:52.959403 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.959348 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 03 15:40:52.959403 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.959360 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 03 15:40:52.959403 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.959369 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 03 15:40:52.959403 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.959377 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 03 15:40:52.959403 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.959384 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 03 15:40:52.959403 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.959403 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 03 15:40:52.959649 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.959413 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 03 15:40:52.959649 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.959425 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 03 15:40:52.959649 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.959434 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 03 15:40:52.960827 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.960816 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 03 15:40:52.960880 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.960831 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 03 15:40:52.961140 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:52.961119 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 03 15:40:52.961229 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:52.961177 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-129.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 03 15:40:52.964217 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.964170 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x5zxc" Jun 03 15:40:52.964367 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.964353 2578 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-129.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 03 15:40:52.964739 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.964724 2578 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 03 15:40:52.964775 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.964769 2578 server.go:1295] "Started kubelet" Jun 03 15:40:52.964887 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.964852 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 03 15:40:52.965041 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.964851 2578 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 03 15:40:52.965097 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.965064 2578 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 03 15:40:52.965684 ip-10-0-129-129 systemd[1]: Started Kubernetes Kubelet. Jun 03 15:40:52.966285 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.966224 2578 server.go:317] "Adding debug handlers to kubelet server" Jun 03 15:40:52.966345 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.966302 2578 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 03 15:40:52.971354 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.971327 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 03 15:40:52.971486 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:52.970676 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-129.ec2.internal.18b59c74c119eeaf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-129.ec2.internal,UID:ip-10-0-129-129.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-129.ec2.internal,},FirstTimestamp:2026-06-03 15:40:52.964740783 +0000 UTC m=+0.349807992,LastTimestamp:2026-06-03 15:40:52.964740783 +0000 UTC m=+0.349807992,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-129.ec2.internal,}" Jun 03 15:40:52.971736 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.971721 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 03 15:40:52.971813 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.971786 2578 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 03 15:40:52.971865 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.971816 2578 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 03 15:40:52.971981 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.971962 2578 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 03 15:40:52.972077 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.972031 2578 reconstruct.go:97] "Volume reconstruction finished" Jun 03 15:40:52.972077 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.972040 2578 reconciler.go:26] "Reconciler: start to sync state" Jun 03 15:40:52.972295 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:52.972273 2578 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 03 15:40:52.972457 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:52.972429 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:52.972533 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.972460 2578 factory.go:55] Registering systemd factory Jun 03 15:40:52.972533 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.972481 2578 factory.go:223] Registration of the systemd container factory successfully Jun 03 15:40:52.972705 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.972681 2578 factory.go:153] Registering CRI-O factory Jun 03 15:40:52.972705 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.972698 2578 factory.go:223] Registration of the crio container factory successfully Jun 03 15:40:52.972804 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.972739 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 03 15:40:52.972804 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.972754 2578 factory.go:103] Registering Raw factory Jun 03 15:40:52.972804 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.972765 2578 manager.go:1196] Started watching for new ooms in manager Jun 03 15:40:52.973415 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.973398 2578 manager.go:319] Starting recovery of all containers Jun 03 15:40:52.973501 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.973427 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x5zxc" Jun 03 15:40:52.976132 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:52.976104 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-129.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 03 15:40:52.976265 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:52.976242 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 03 15:40:52.985465 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.985272 2578 manager.go:324] Recovery completed Jun 03 15:40:52.986617 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:52.986600 2578 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jun 03 15:40:52.989575 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.989563 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 15:40:52.991339 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.991319 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasSufficientMemory" Jun 03 15:40:52.991409 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.991353 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 15:40:52.991409 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.991363 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasSufficientPID" Jun 03 15:40:52.991856 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.991839 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 03 15:40:52.991856 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.991855 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 03 15:40:52.991996 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.991874 2578 state_mem.go:36] "Initialized new in-memory state store" Jun 03 15:40:52.993981 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.993967 2578 policy_none.go:49] "None policy: Start" Jun 03 15:40:52.994064 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.993985 2578 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 03 15:40:52.994064 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:52.993998 2578 state_mem.go:35] "Initializing new in-memory state store" Jun 03 15:40:53.034843 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.034822 2578 manager.go:341] "Starting Device Plugin manager" Jun 03 15:40:53.041605 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.034871 2578 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 03 15:40:53.041605 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.034896 2578 server.go:85] "Starting device plugin registration server" Jun 03 15:40:53.041605 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.035169 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 03 15:40:53.041605 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.035179 2578 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 03 15:40:53.041605 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.035309 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 03 15:40:53.041605 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.035406 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 03 15:40:53.041605 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.035435 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 03 15:40:53.041605 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.035912 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 03 15:40:53.041605 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.035949 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:53.105807 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.105760 2578 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 03 15:40:53.106954 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.106936 2578 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 03 15:40:53.106954 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.106960 2578 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 03 15:40:53.107136 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.106984 2578 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 03 15:40:53.107136 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.106991 2578 kubelet.go:2451] "Starting kubelet main sync loop" Jun 03 15:40:53.107136 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.107033 2578 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 03 15:40:53.110455 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.110437 2578 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 15:40:53.135591 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.135510 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 15:40:53.136523 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.136507 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasSufficientMemory" Jun 03 15:40:53.136614 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.136535 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 15:40:53.136614 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.136545 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasSufficientPID" Jun 03 15:40:53.136614 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.136571 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.143556 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.143540 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.143615 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.143565 2578 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-129.ec2.internal\": node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:53.161093 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.161071 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:53.207657 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.207614 2578 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-129.ec2.internal"] Jun 03 15:40:53.207767 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.207702 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 15:40:53.208634 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.208620 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasSufficientMemory" Jun 03 15:40:53.208687 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.208647 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 15:40:53.208687 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.208659 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasSufficientPID" Jun 03 15:40:53.209896 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.209879 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 15:40:53.210090 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.210075 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.210137 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.210121 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 15:40:53.210612 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.210597 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasSufficientMemory" Jun 03 15:40:53.210714 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.210623 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 15:40:53.210714 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.210638 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasSufficientPID" Jun 03 15:40:53.210714 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.210666 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasSufficientMemory" Jun 03 15:40:53.210714 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.210687 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 15:40:53.210714 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.210696 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasSufficientPID" Jun 03 15:40:53.211742 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.211726 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.211829 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.211749 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 15:40:53.212366 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.212350 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasSufficientMemory" Jun 03 15:40:53.212366 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.212377 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 15:40:53.212502 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.212389 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeHasSufficientPID" Jun 03 15:40:53.245991 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.245973 2578 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-129.ec2.internal\" not found" node="ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.250396 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.250379 2578 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-129.ec2.internal\" not found" node="ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.261298 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.261283 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:53.273585 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.273567 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94971eb58eb2e55ff58107d4a8f407cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal\" (UID: \"94971eb58eb2e55ff58107d4a8f407cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.273663 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.273595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94971eb58eb2e55ff58107d4a8f407cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal\" (UID: \"94971eb58eb2e55ff58107d4a8f407cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.273663 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.273622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/43488e4976b4b669cc3a2c15df77a88a-config\") pod \"kube-apiserver-proxy-ip-10-0-129-129.ec2.internal\" (UID: \"43488e4976b4b669cc3a2c15df77a88a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.361921 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.361883 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:53.374296 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.374277 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94971eb58eb2e55ff58107d4a8f407cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal\" (UID: \"94971eb58eb2e55ff58107d4a8f407cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.374358 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.374302 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94971eb58eb2e55ff58107d4a8f407cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal\" (UID: \"94971eb58eb2e55ff58107d4a8f407cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.374358 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.374322 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/43488e4976b4b669cc3a2c15df77a88a-config\") pod \"kube-apiserver-proxy-ip-10-0-129-129.ec2.internal\" (UID: \"43488e4976b4b669cc3a2c15df77a88a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.374358 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.374356 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94971eb58eb2e55ff58107d4a8f407cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal\" (UID: \"94971eb58eb2e55ff58107d4a8f407cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.374448 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.374368 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94971eb58eb2e55ff58107d4a8f407cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal\" (UID: \"94971eb58eb2e55ff58107d4a8f407cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.374448 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.374406 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/43488e4976b4b669cc3a2c15df77a88a-config\") pod \"kube-apiserver-proxy-ip-10-0-129-129.ec2.internal\" (UID: \"43488e4976b4b669cc3a2c15df77a88a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.462722 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.462650 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:53.548237 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.548188 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.553755 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.553737 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-129.ec2.internal" Jun 03 15:40:53.563388 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.563369 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:53.663900 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.663861 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:53.764438 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.764359 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:53.861978 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.861948 2578 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 15:40:53.865064 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.865044 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:53.893482 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.893463 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 03 15:40:53.893922 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.893578 2578 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 15:40:53.893922 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.893615 2578 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 15:40:53.965277 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:53.965176 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:53.971718 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.971696 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 03 15:40:53.975193 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.975167 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-02 15:35:52 +0000 UTC" deadline="2027-11-25 14:14:57.360233932 +0000 UTC" Jun 03 15:40:53.975300 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.975192 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12958h34m3.385044861s" Jun 03 15:40:53.986491 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:53.986472 2578 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 15:40:54.008943 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.008916 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5lsjs" Jun 03 15:40:54.017353 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.017295 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5lsjs" Jun 03 15:40:54.065930 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:54.065890 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:54.082250 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:54.082193 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43488e4976b4b669cc3a2c15df77a88a.slice/crio-5e20e18879ae7aff363e386ec235906aa5e8d37b618905651c3a9ef1f559c202 WatchSource:0}: Error finding container 5e20e18879ae7aff363e386ec235906aa5e8d37b618905651c3a9ef1f559c202: Status 404 returned error can't find the container with id 5e20e18879ae7aff363e386ec235906aa5e8d37b618905651c3a9ef1f559c202 Jun 03 15:40:54.082871 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:54.082852 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94971eb58eb2e55ff58107d4a8f407cc.slice/crio-191302e5d45810ac6ea20efc02c0e112e5e841ae1622bb066bc3abc357619fd4 WatchSource:0}: Error finding container 191302e5d45810ac6ea20efc02c0e112e5e841ae1622bb066bc3abc357619fd4: Status 404 returned error can't find the container with id 191302e5d45810ac6ea20efc02c0e112e5e841ae1622bb066bc3abc357619fd4 Jun 03 15:40:54.087300 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.087282 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 15:40:54.109815 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.109767 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" event={"ID":"94971eb58eb2e55ff58107d4a8f407cc","Type":"ContainerStarted","Data":"191302e5d45810ac6ea20efc02c0e112e5e841ae1622bb066bc3abc357619fd4"} Jun 03 15:40:54.110650 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.110629 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-129.ec2.internal" event={"ID":"43488e4976b4b669cc3a2c15df77a88a","Type":"ContainerStarted","Data":"5e20e18879ae7aff363e386ec235906aa5e8d37b618905651c3a9ef1f559c202"} Jun 03 15:40:54.166835 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:54.166803 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-129.ec2.internal\" not found" Jun 03 15:40:54.178334 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.178309 2578 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 15:40:54.272418 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.272357 2578 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" Jun 03 15:40:54.283399 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.283380 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 15:40:54.284811 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.284799 2578 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-129.ec2.internal" Jun 03 15:40:54.292716 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.292695 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 15:40:54.360945 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.360919 2578 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 15:40:54.953416 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.953382 2578 apiserver.go:52] "Watching apiserver" Jun 03 15:40:54.954893 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.954867 2578 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 15:40:54.959702 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.959540 2578 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 03 15:40:54.960000 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.959972 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-v6zjk","openshift-multus/network-metrics-daemon-t7kx5","openshift-network-diagnostics/network-check-target-hm6g8","openshift-cluster-node-tuning-operator/tuned-p9b7b","openshift-dns/node-resolver-4hx4n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal","openshift-network-operator/iptables-alerter-x74fd","openshift-ovn-kubernetes/ovnkube-node-kdw5d","kube-system/konnectivity-agent-4dfqm","kube-system/kube-apiserver-proxy-ip-10-0-129-129.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8","openshift-image-registry/node-ca-gh9z6","openshift-multus/multus-additional-cni-plugins-4jmlt"] Jun 03 15:40:54.961543 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.961521 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x74fd" Jun 03 15:40:54.962620 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.962600 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:40:54.962737 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:54.962674 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:40:54.963642 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.963602 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:40:54.963741 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:54.963680 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:40:54.963970 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.963948 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 03 15:40:54.964099 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.964016 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 03 15:40:54.964167 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.964124 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 03 15:40:54.964230 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.964174 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-g6gw5\"" Jun 03 15:40:54.964600 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.964582 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.966147 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.965846 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4hx4n" Jun 03 15:40:54.966886 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.966841 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 03 15:40:54.966982 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.966943 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7nfm7\"" Jun 03 15:40:54.967433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.967414 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 03 15:40:54.967749 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.967724 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.968023 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.968005 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 03 15:40:54.968113 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.968062 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pzdzn\"" Jun 03 15:40:54.968398 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.968382 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 03 15:40:54.968721 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.968702 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.969881 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.969722 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 03 15:40:54.969881 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.969849 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 03 15:40:54.970147 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.970051 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nrd7q\"" Jun 03 15:40:54.970147 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.970082 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4dfqm" Jun 03 15:40:54.970799 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.970636 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 03 15:40:54.970799 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.970723 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 03 15:40:54.970938 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.970848 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 03 15:40:54.970992 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.970964 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 03 15:40:54.971441 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.971229 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hdzfd\"" Jun 03 15:40:54.971441 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.971261 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 03 15:40:54.971441 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.971338 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 03 15:40:54.971903 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.971777 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 03 15:40:54.971903 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.971790 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 03 15:40:54.972184 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.972167 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 03 15:40:54.972419 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.972399 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-64q26\"" Jun 03 15:40:54.972597 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.972579 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 03 15:40:54.973214 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.972833 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gh9z6" Jun 03 15:40:54.973214 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.972930 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:54.974260 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.974239 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:54.974904 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.974756 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 03 15:40:54.974904 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.974810 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 03 15:40:54.975066 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.975048 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 03 15:40:54.975127 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.975070 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m6nxx\"" Jun 03 15:40:54.976012 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.975798 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 03 15:40:54.976012 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.975838 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 03 15:40:54.976012 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.975837 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 03 15:40:54.976477 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.976364 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 03 15:40:54.976477 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.976438 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 03 15:40:54.978637 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.977235 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-x7sgw\"" Jun 03 15:40:54.980263 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.978891 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mxb28\"" Jun 03 15:40:54.984442 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzptj\" (UniqueName: \"kubernetes.io/projected/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-kube-api-access-fzptj\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:40:54.984554 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-node-log\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.984554 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-sysctl-conf\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.984554 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984510 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-host\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.984554 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984532 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-tuned\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.984554 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984549 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:54.984768 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5f1cad3-acab-45ec-8328-0161d1e9942a-iptables-alerter-script\") pod \"iptables-alerter-x74fd\" (UID: \"e5f1cad3-acab-45ec-8328-0161d1e9942a\") " pod="openshift-network-operator/iptables-alerter-x74fd" Jun 03 15:40:54.984768 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984630 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-multus-cni-dir\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.984768 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984657 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw929\" (UniqueName: \"kubernetes.io/projected/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-kube-api-access-xw929\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.984768 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984695 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfnr4\" (UniqueName: \"kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4\") pod \"network-check-target-hm6g8\" (UID: \"43da5244-85e8-4634-8827-e268eb3df28b\") " pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:40:54.984768 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-kubernetes\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.984986 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984773 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-sys-fs\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:54.984986 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984808 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22b6b156-cdf5-4696-b64c-fec58fcc28e5-host\") pod \"node-ca-gh9z6\" (UID: \"22b6b156-cdf5-4696-b64c-fec58fcc28e5\") " pod="openshift-image-registry/node-ca-gh9z6" Jun 03 15:40:54.984986 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984846 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-run-multus-certs\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.984986 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984872 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-systemd\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.984986 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984895 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-run-k8s-cni-cncf-io\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.984986 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984916 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-slash\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.984986 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984941 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/13e88df3-970c-472f-9f44-1a2102fa3695-ovn-node-metrics-cert\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.984986 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984964 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e3c78079-acba-48d7-a32a-2dd02cb04d98-hosts-file\") pod \"node-resolver-4hx4n\" (UID: \"e3c78079-acba-48d7-a32a-2dd02cb04d98\") " pod="openshift-dns/node-resolver-4hx4n" Jun 03 15:40:54.984986 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.984986 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-registration-dir\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985007 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-device-dir\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985029 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-cnibin\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985050 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-kubelet\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985067 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-run-netns\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985087 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-etc-selinux\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985119 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/22b6b156-cdf5-4696-b64c-fec58fcc28e5-serviceca\") pod \"node-ca-gh9z6\" (UID: \"22b6b156-cdf5-4696-b64c-fec58fcc28e5\") " pod="openshift-image-registry/node-ca-gh9z6" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985140 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-sysconfig\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985154 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-var-lib-kubelet\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985168 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-run-openvswitch\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985190 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-run-ovn-kubernetes\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985228 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-cni-netd\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985245 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-cni-bin\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985258 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-run\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985278 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-socket-dir\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985302 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-run-systemd\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985335 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/13e88df3-970c-472f-9f44-1a2102fa3695-env-overrides\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985394 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985365 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-sys\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985410 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-lib-modules\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985445 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qwl4\" (UniqueName: \"kubernetes.io/projected/e5f1cad3-acab-45ec-8328-0161d1e9942a-kube-api-access-7qwl4\") pod \"iptables-alerter-x74fd\" (UID: \"e5f1cad3-acab-45ec-8328-0161d1e9942a\") " pod="openshift-network-operator/iptables-alerter-x74fd" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-multus-conf-dir\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985513 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-var-lib-openvswitch\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985554 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hj8j\" (UniqueName: \"kubernetes.io/projected/13e88df3-970c-472f-9f44-1a2102fa3695-kube-api-access-5hj8j\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985597 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985621 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-system-cni-dir\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985651 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-multus-socket-dir-parent\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985681 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-systemd-units\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985704 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-log-socket\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985746 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-sysctl-d\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h9pr\" (UniqueName: \"kubernetes.io/projected/e3c78079-acba-48d7-a32a-2dd02cb04d98-kube-api-access-2h9pr\") pod \"node-resolver-4hx4n\" (UID: \"e3c78079-acba-48d7-a32a-2dd02cb04d98\") " pod="openshift-dns/node-resolver-4hx4n" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985801 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-run-netns\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985824 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-run-ovn\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985849 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwkbc\" (UniqueName: \"kubernetes.io/projected/22b6b156-cdf5-4696-b64c-fec58fcc28e5-kube-api-access-fwkbc\") pod \"node-ca-gh9z6\" (UID: \"22b6b156-cdf5-4696-b64c-fec58fcc28e5\") " pod="openshift-image-registry/node-ca-gh9z6" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985903 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-os-release\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.985927 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985928 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-cni-binary-copy\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985953 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-etc-kubernetes\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.985979 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-var-lib-cni-multus\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986002 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-etc-openvswitch\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986025 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/13e88df3-970c-472f-9f44-1a2102fa3695-ovnkube-script-lib\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986051 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a239c904-33ce-4c96-88ce-c976799776e8-konnectivity-ca\") pod \"konnectivity-agent-4dfqm\" (UID: \"a239c904-33ce-4c96-88ce-c976799776e8\") " pod="kube-system/konnectivity-agent-4dfqm" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986074 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-multus-daemon-config\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986101 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986125 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a239c904-33ce-4c96-88ce-c976799776e8-agent-certs\") pod \"konnectivity-agent-4dfqm\" (UID: \"a239c904-33ce-4c96-88ce-c976799776e8\") " pod="kube-system/konnectivity-agent-4dfqm" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-modprobe-d\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986228 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a795c9e9-6262-491f-b6e3-db2a50aea8eb-tmp\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986255 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-var-lib-cni-bin\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986277 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-hostroot\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986331 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/13e88df3-970c-472f-9f44-1a2102fa3695-ovnkube-config\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-var-lib-kubelet\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986437 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhjdp\" (UniqueName: \"kubernetes.io/projected/a795c9e9-6262-491f-b6e3-db2a50aea8eb-kube-api-access-qhjdp\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:54.986618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986473 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3c78079-acba-48d7-a32a-2dd02cb04d98-tmp-dir\") pod \"node-resolver-4hx4n\" (UID: \"e3c78079-acba-48d7-a32a-2dd02cb04d98\") " pod="openshift-dns/node-resolver-4hx4n" Jun 03 15:40:54.987288 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986514 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnlm5\" (UniqueName: \"kubernetes.io/projected/f94fa647-de64-4615-8168-d3521878583e-kube-api-access-vnlm5\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:54.987288 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:54.986555 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5f1cad3-acab-45ec-8328-0161d1e9942a-host-slash\") pod \"iptables-alerter-x74fd\" (UID: \"e5f1cad3-acab-45ec-8328-0161d1e9942a\") " pod="openshift-network-operator/iptables-alerter-x74fd" Jun 03 15:40:55.017983 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.017951 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 15:35:54 +0000 UTC" deadline="2028-01-11 11:06:01.302947223 +0000 UTC" Jun 03 15:40:55.017983 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.017983 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14083h25m6.284967671s" Jun 03 15:40:55.072862 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.072828 2578 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 03 15:40:55.087247 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087215 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-socket-dir\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.087398 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087258 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-run-systemd\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.087398 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087276 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/13e88df3-970c-472f-9f44-1a2102fa3695-env-overrides\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.087398 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087295 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-sys\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.087398 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087316 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-lib-modules\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.087398 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087349 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-run-systemd\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.087398 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087345 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a98bd1db-934b-442e-8851-e8a60695aa8d-system-cni-dir\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.087398 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087385 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7qwl4\" (UniqueName: \"kubernetes.io/projected/e5f1cad3-acab-45ec-8328-0161d1e9942a-kube-api-access-7qwl4\") pod \"iptables-alerter-x74fd\" (UID: \"e5f1cad3-acab-45ec-8328-0161d1e9942a\") " pod="openshift-network-operator/iptables-alerter-x74fd" Jun 03 15:40:55.087398 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087390 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-sys\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087403 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-socket-dir\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087412 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-multus-conf-dir\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087433 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-var-lib-openvswitch\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087456 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5hj8j\" (UniqueName: \"kubernetes.io/projected/13e88df3-970c-472f-9f44-1a2102fa3695-kube-api-access-5hj8j\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087466 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-multus-conf-dir\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a98bd1db-934b-442e-8851-e8a60695aa8d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087493 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-lib-modules\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087510 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087515 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-var-lib-openvswitch\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087541 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-system-cni-dir\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087568 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-multus-socket-dir-parent\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087591 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-systemd-units\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:55.087598 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087613 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-log-socket\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087639 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-sysctl-d\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:55.087693 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs podName:a9d8c3c0-82e0-48b5-909a-a7887d92baa9 nodeName:}" failed. No retries permitted until 2026-06-03 15:40:55.587646267 +0000 UTC m=+2.972713480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs") pod "network-metrics-daemon-t7kx5" (UID: "a9d8c3c0-82e0-48b5-909a-a7887d92baa9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:40:55.087783 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087705 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-multus-socket-dir-parent\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087738 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-sysctl-d\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087755 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-system-cni-dir\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087754 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2h9pr\" (UniqueName: \"kubernetes.io/projected/e3c78079-acba-48d7-a32a-2dd02cb04d98-kube-api-access-2h9pr\") pod \"node-resolver-4hx4n\" (UID: \"e3c78079-acba-48d7-a32a-2dd02cb04d98\") " pod="openshift-dns/node-resolver-4hx4n" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087777 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-systemd-units\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087796 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a98bd1db-934b-442e-8851-e8a60695aa8d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087816 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/13e88df3-970c-472f-9f44-1a2102fa3695-env-overrides\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087826 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-run-netns\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087832 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-log-socket\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087851 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-run-ovn\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087877 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fwkbc\" (UniqueName: \"kubernetes.io/projected/22b6b156-cdf5-4696-b64c-fec58fcc28e5-kube-api-access-fwkbc\") pod \"node-ca-gh9z6\" (UID: \"22b6b156-cdf5-4696-b64c-fec58fcc28e5\") " pod="openshift-image-registry/node-ca-gh9z6" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087887 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-run-netns\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087901 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-os-release\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087906 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-run-ovn\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087926 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-cni-binary-copy\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087974 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-os-release\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.087974 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-etc-kubernetes\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088010 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-etc-kubernetes\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.088572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088022 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a98bd1db-934b-442e-8851-e8a60695aa8d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088053 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-var-lib-cni-multus\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088109 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-var-lib-cni-multus\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088080 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-etc-openvswitch\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088158 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-etc-openvswitch\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088171 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/13e88df3-970c-472f-9f44-1a2102fa3695-ovnkube-script-lib\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088214 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a239c904-33ce-4c96-88ce-c976799776e8-konnectivity-ca\") pod \"konnectivity-agent-4dfqm\" (UID: \"a239c904-33ce-4c96-88ce-c976799776e8\") " pod="kube-system/konnectivity-agent-4dfqm" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088243 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-multus-daemon-config\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088271 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088297 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a239c904-33ce-4c96-88ce-c976799776e8-agent-certs\") pod \"konnectivity-agent-4dfqm\" (UID: \"a239c904-33ce-4c96-88ce-c976799776e8\") " pod="kube-system/konnectivity-agent-4dfqm" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088322 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-modprobe-d\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088346 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a795c9e9-6262-491f-b6e3-db2a50aea8eb-tmp\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088362 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088374 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a98bd1db-934b-442e-8851-e8a60695aa8d-cni-binary-copy\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088396 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-cni-binary-copy\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088401 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-var-lib-cni-bin\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088434 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-var-lib-cni-bin\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.089433 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088444 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-hostroot\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088469 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/13e88df3-970c-472f-9f44-1a2102fa3695-ovnkube-config\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088497 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-var-lib-kubelet\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088523 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qhjdp\" (UniqueName: \"kubernetes.io/projected/a795c9e9-6262-491f-b6e3-db2a50aea8eb-kube-api-access-qhjdp\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088540 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-modprobe-d\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088546 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3c78079-acba-48d7-a32a-2dd02cb04d98-tmp-dir\") pod \"node-resolver-4hx4n\" (UID: \"e3c78079-acba-48d7-a32a-2dd02cb04d98\") " pod="openshift-dns/node-resolver-4hx4n" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088573 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vnlm5\" (UniqueName: \"kubernetes.io/projected/f94fa647-de64-4615-8168-d3521878583e-kube-api-access-vnlm5\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088601 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5f1cad3-acab-45ec-8328-0161d1e9942a-host-slash\") pod \"iptables-alerter-x74fd\" (UID: \"e5f1cad3-acab-45ec-8328-0161d1e9942a\") " pod="openshift-network-operator/iptables-alerter-x74fd" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088626 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fzptj\" (UniqueName: \"kubernetes.io/projected/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-kube-api-access-fzptj\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088630 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088651 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-node-log\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088672 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-sysctl-conf\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088694 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-host\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088718 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-tuned\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088742 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088767 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5f1cad3-acab-45ec-8328-0161d1e9942a-iptables-alerter-script\") pod \"iptables-alerter-x74fd\" (UID: \"e5f1cad3-acab-45ec-8328-0161d1e9942a\") " pod="openshift-network-operator/iptables-alerter-x74fd" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088774 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/13e88df3-970c-472f-9f44-1a2102fa3695-ovnkube-script-lib\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088785 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a239c904-33ce-4c96-88ce-c976799776e8-konnectivity-ca\") pod \"konnectivity-agent-4dfqm\" (UID: \"a239c904-33ce-4c96-88ce-c976799776e8\") " pod="kube-system/konnectivity-agent-4dfqm" Jun 03 15:40:55.090171 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088793 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-multus-cni-dir\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088842 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-multus-daemon-config\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088849 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-multus-cni-dir\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088843 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xw929\" (UniqueName: \"kubernetes.io/projected/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-kube-api-access-xw929\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088866 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5f1cad3-acab-45ec-8328-0161d1e9942a-host-slash\") pod \"iptables-alerter-x74fd\" (UID: \"e5f1cad3-acab-45ec-8328-0161d1e9942a\") " pod="openshift-network-operator/iptables-alerter-x74fd" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088893 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-hostroot\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088894 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnr4\" (UniqueName: \"kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4\") pod \"network-check-target-hm6g8\" (UID: \"43da5244-85e8-4634-8827-e268eb3df28b\") " pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088933 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-kubernetes\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088959 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-sys-fs\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088987 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22b6b156-cdf5-4696-b64c-fec58fcc28e5-host\") pod \"node-ca-gh9z6\" (UID: \"22b6b156-cdf5-4696-b64c-fec58fcc28e5\") " pod="openshift-image-registry/node-ca-gh9z6" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089015 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a98bd1db-934b-442e-8851-e8a60695aa8d-os-release\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089042 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-sysctl-conf\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089045 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-run-multus-certs\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.088895 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-var-lib-kubelet\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089081 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-systemd\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089115 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089126 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-run-k8s-cni-cncf-io\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089153 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-slash\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.090967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089288 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-run-k8s-cni-cncf-io\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089328 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3c78079-acba-48d7-a32a-2dd02cb04d98-tmp-dir\") pod \"node-resolver-4hx4n\" (UID: \"e3c78079-acba-48d7-a32a-2dd02cb04d98\") " pod="openshift-dns/node-resolver-4hx4n" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089336 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22b6b156-cdf5-4696-b64c-fec58fcc28e5-host\") pod \"node-ca-gh9z6\" (UID: \"22b6b156-cdf5-4696-b64c-fec58fcc28e5\") " pod="openshift-image-registry/node-ca-gh9z6" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089348 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-host\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089392 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-slash\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089417 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-kubernetes\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089462 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-sys-fs\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089502 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-systemd\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089630 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-run-multus-certs\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089178 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/13e88df3-970c-472f-9f44-1a2102fa3695-ovn-node-metrics-cert\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089732 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e3c78079-acba-48d7-a32a-2dd02cb04d98-hosts-file\") pod \"node-resolver-4hx4n\" (UID: \"e3c78079-acba-48d7-a32a-2dd02cb04d98\") " pod="openshift-dns/node-resolver-4hx4n" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089768 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5f1cad3-acab-45ec-8328-0161d1e9942a-iptables-alerter-script\") pod \"iptables-alerter-x74fd\" (UID: \"e5f1cad3-acab-45ec-8328-0161d1e9942a\") " pod="openshift-network-operator/iptables-alerter-x74fd" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089780 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-registration-dir\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089814 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-device-dir\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089848 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e3c78079-acba-48d7-a32a-2dd02cb04d98-hosts-file\") pod \"node-resolver-4hx4n\" (UID: \"e3c78079-acba-48d7-a32a-2dd02cb04d98\") " pod="openshift-dns/node-resolver-4hx4n" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089853 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/13e88df3-970c-472f-9f44-1a2102fa3695-ovnkube-config\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a98bd1db-934b-442e-8851-e8a60695aa8d-cnibin\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.091800 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089899 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn6hd\" (UniqueName: \"kubernetes.io/projected/a98bd1db-934b-442e-8851-e8a60695aa8d-kube-api-access-rn6hd\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089906 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-registration-dir\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089912 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-node-log\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089926 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-cnibin\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089958 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-device-dir\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089959 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-kubelet\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.089997 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-run-netns\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090007 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-cnibin\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090022 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-etc-selinux\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090044 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-kubelet\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090049 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/22b6b156-cdf5-4696-b64c-fec58fcc28e5-serviceca\") pod \"node-ca-gh9z6\" (UID: \"22b6b156-cdf5-4696-b64c-fec58fcc28e5\") " pod="openshift-image-registry/node-ca-gh9z6" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090086 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-sysconfig\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090109 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-run-netns\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090147 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-var-lib-kubelet\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090150 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-sysconfig\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090111 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-host-var-lib-kubelet\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090185 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-run-openvswitch\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090227 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-run-ovn-kubernetes\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.092580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090241 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f94fa647-de64-4615-8168-d3521878583e-etc-selinux\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.093377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090251 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-cni-netd\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.093377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090278 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-cni-bin\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.093377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090283 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-run-openvswitch\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.093377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090300 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-run\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.093377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090319 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-cni-netd\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.093377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090352 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-run-ovn-kubernetes\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.093377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090367 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a795c9e9-6262-491f-b6e3-db2a50aea8eb-run\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.093377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090386 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13e88df3-970c-472f-9f44-1a2102fa3695-host-cni-bin\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.093377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.090431 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/22b6b156-cdf5-4696-b64c-fec58fcc28e5-serviceca\") pod \"node-ca-gh9z6\" (UID: \"22b6b156-cdf5-4696-b64c-fec58fcc28e5\") " pod="openshift-image-registry/node-ca-gh9z6" Jun 03 15:40:55.093377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.092058 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a795c9e9-6262-491f-b6e3-db2a50aea8eb-tmp\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.093377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.092096 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a795c9e9-6262-491f-b6e3-db2a50aea8eb-etc-tuned\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.093377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.092338 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/13e88df3-970c-472f-9f44-1a2102fa3695-ovn-node-metrics-cert\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.093761 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.093489 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a239c904-33ce-4c96-88ce-c976799776e8-agent-certs\") pod \"konnectivity-agent-4dfqm\" (UID: \"a239c904-33ce-4c96-88ce-c976799776e8\") " pod="kube-system/konnectivity-agent-4dfqm" Jun 03 15:40:55.098768 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:55.098744 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:40:55.098768 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:55.098768 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:40:55.098923 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:55.098791 2578 projected.go:194] Error preparing data for projected volume kube-api-access-bfnr4 for pod openshift-network-diagnostics/network-check-target-hm6g8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:40:55.098923 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:55.098868 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4 podName:43da5244-85e8-4634-8827-e268eb3df28b nodeName:}" failed. No retries permitted until 2026-06-03 15:40:55.598841287 +0000 UTC m=+2.983908485 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bfnr4" (UniqueName: "kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4") pod "network-check-target-hm6g8" (UID: "43da5244-85e8-4634-8827-e268eb3df28b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:40:55.099033 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.098996 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h9pr\" (UniqueName: \"kubernetes.io/projected/e3c78079-acba-48d7-a32a-2dd02cb04d98-kube-api-access-2h9pr\") pod \"node-resolver-4hx4n\" (UID: \"e3c78079-acba-48d7-a32a-2dd02cb04d98\") " pod="openshift-dns/node-resolver-4hx4n" Jun 03 15:40:55.099824 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.099413 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qwl4\" (UniqueName: \"kubernetes.io/projected/e5f1cad3-acab-45ec-8328-0161d1e9942a-kube-api-access-7qwl4\") pod \"iptables-alerter-x74fd\" (UID: \"e5f1cad3-acab-45ec-8328-0161d1e9942a\") " pod="openshift-network-operator/iptables-alerter-x74fd" Jun 03 15:40:55.101708 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.101670 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhjdp\" (UniqueName: \"kubernetes.io/projected/a795c9e9-6262-491f-b6e3-db2a50aea8eb-kube-api-access-qhjdp\") pod \"tuned-p9b7b\" (UID: \"a795c9e9-6262-491f-b6e3-db2a50aea8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.102111 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.102090 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw929\" (UniqueName: \"kubernetes.io/projected/b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e-kube-api-access-xw929\") pod \"multus-v6zjk\" (UID: \"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e\") " pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.102528 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.102504 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzptj\" (UniqueName: \"kubernetes.io/projected/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-kube-api-access-fzptj\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:40:55.103939 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.103913 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hj8j\" (UniqueName: \"kubernetes.io/projected/13e88df3-970c-472f-9f44-1a2102fa3695-kube-api-access-5hj8j\") pod \"ovnkube-node-kdw5d\" (UID: \"13e88df3-970c-472f-9f44-1a2102fa3695\") " pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.104487 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.104460 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnlm5\" (UniqueName: \"kubernetes.io/projected/f94fa647-de64-4615-8168-d3521878583e-kube-api-access-vnlm5\") pod \"aws-ebs-csi-driver-node-jr2z8\" (UID: \"f94fa647-de64-4615-8168-d3521878583e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.104846 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.104830 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwkbc\" (UniqueName: \"kubernetes.io/projected/22b6b156-cdf5-4696-b64c-fec58fcc28e5-kube-api-access-fwkbc\") pod \"node-ca-gh9z6\" (UID: \"22b6b156-cdf5-4696-b64c-fec58fcc28e5\") " pod="openshift-image-registry/node-ca-gh9z6" Jun 03 15:40:55.191418 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.191387 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a98bd1db-934b-442e-8851-e8a60695aa8d-system-cni-dir\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.191418 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.191422 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a98bd1db-934b-442e-8851-e8a60695aa8d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.191653 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.191464 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a98bd1db-934b-442e-8851-e8a60695aa8d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.191653 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.191486 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a98bd1db-934b-442e-8851-e8a60695aa8d-system-cni-dir\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.191653 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.191493 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a98bd1db-934b-442e-8851-e8a60695aa8d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.191653 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.191530 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a98bd1db-934b-442e-8851-e8a60695aa8d-cni-binary-copy\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.191653 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.191613 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a98bd1db-934b-442e-8851-e8a60695aa8d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.191653 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.191639 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a98bd1db-934b-442e-8851-e8a60695aa8d-os-release\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.191936 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.191693 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a98bd1db-934b-442e-8851-e8a60695aa8d-cnibin\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.191936 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.191726 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rn6hd\" (UniqueName: \"kubernetes.io/projected/a98bd1db-934b-442e-8851-e8a60695aa8d-kube-api-access-rn6hd\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.191936 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.191808 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a98bd1db-934b-442e-8851-e8a60695aa8d-cnibin\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.191936 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.191829 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a98bd1db-934b-442e-8851-e8a60695aa8d-os-release\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.192121 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.192047 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a98bd1db-934b-442e-8851-e8a60695aa8d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.192240 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.192221 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a98bd1db-934b-442e-8851-e8a60695aa8d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.192277 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.192256 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a98bd1db-934b-442e-8851-e8a60695aa8d-cni-binary-copy\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.201330 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.201308 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn6hd\" (UniqueName: \"kubernetes.io/projected/a98bd1db-934b-442e-8851-e8a60695aa8d-kube-api-access-rn6hd\") pod \"multus-additional-cni-plugins-4jmlt\" (UID: \"a98bd1db-934b-442e-8851-e8a60695aa8d\") " pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.285457 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.285374 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x74fd" Jun 03 15:40:55.291034 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.291013 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" Jun 03 15:40:55.297708 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.297686 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4hx4n" Jun 03 15:40:55.303346 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.303319 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v6zjk" Jun 03 15:40:55.309603 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.308940 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:40:55.315959 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.315942 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4dfqm" Jun 03 15:40:55.321482 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.321464 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gh9z6" Jun 03 15:40:55.328025 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.328005 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" Jun 03 15:40:55.333570 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.333553 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4jmlt" Jun 03 15:40:55.594643 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.594552 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:40:55.594803 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:55.594678 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:40:55.594803 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:55.594743 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs podName:a9d8c3c0-82e0-48b5-909a-a7887d92baa9 nodeName:}" failed. No retries permitted until 2026-06-03 15:40:56.594724271 +0000 UTC m=+3.979791470 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs") pod "network-metrics-daemon-t7kx5" (UID: "a9d8c3c0-82e0-48b5-909a-a7887d92baa9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:40:55.657155 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:55.657121 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda239c904_33ce_4c96_88ce_c976799776e8.slice/crio-f82d3b7a4171f4f5b415cb3710896d95d52199e1fe1197626906666d18950234 WatchSource:0}: Error finding container f82d3b7a4171f4f5b415cb3710896d95d52199e1fe1197626906666d18950234: Status 404 returned error can't find the container with id f82d3b7a4171f4f5b415cb3710896d95d52199e1fe1197626906666d18950234 Jun 03 15:40:55.658189 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:55.658163 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf94fa647_de64_4615_8168_d3521878583e.slice/crio-f570471f3a473f84846b105a688d961961a24edc1675a01c3b250bc8118fcd3a WatchSource:0}: Error finding container f570471f3a473f84846b105a688d961961a24edc1675a01c3b250bc8118fcd3a: Status 404 returned error can't find the container with id f570471f3a473f84846b105a688d961961a24edc1675a01c3b250bc8118fcd3a Jun 03 15:40:55.661684 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:55.661657 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22b6b156_cdf5_4696_b64c_fec58fcc28e5.slice/crio-0e15ae51525626a3fd0ed0fdb9d87a3b55d03d46d9ba1c10b7190924165b6889 WatchSource:0}: Error finding container 0e15ae51525626a3fd0ed0fdb9d87a3b55d03d46d9ba1c10b7190924165b6889: Status 404 returned error can't find the container with id 0e15ae51525626a3fd0ed0fdb9d87a3b55d03d46d9ba1c10b7190924165b6889 Jun 03 15:40:55.662711 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:55.662689 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda795c9e9_6262_491f_b6e3_db2a50aea8eb.slice/crio-2592ebe28322fb88b7bdf8d9e64f14fe49bfa32630a3e0a56e0d61c9feee5cc0 WatchSource:0}: Error finding container 2592ebe28322fb88b7bdf8d9e64f14fe49bfa32630a3e0a56e0d61c9feee5cc0: Status 404 returned error can't find the container with id 2592ebe28322fb88b7bdf8d9e64f14fe49bfa32630a3e0a56e0d61c9feee5cc0 Jun 03 15:40:55.664683 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:55.664177 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb47a65fc_0f6c_4a56_96eb_fd5065fa8a0e.slice/crio-125a1cfb3e11fbc9bc9765d8f45da129f7da3059bb288d09ed91724342df6214 WatchSource:0}: Error finding container 125a1cfb3e11fbc9bc9765d8f45da129f7da3059bb288d09ed91724342df6214: Status 404 returned error can't find the container with id 125a1cfb3e11fbc9bc9765d8f45da129f7da3059bb288d09ed91724342df6214 Jun 03 15:40:55.665587 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:55.665328 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5f1cad3_acab_45ec_8328_0161d1e9942a.slice/crio-9458054cb2ee6cc463df4d8130820d088387090cf4f4b566c4f60d949d476ceb WatchSource:0}: Error finding container 9458054cb2ee6cc463df4d8130820d088387090cf4f4b566c4f60d949d476ceb: Status 404 returned error can't find the container with id 9458054cb2ee6cc463df4d8130820d088387090cf4f4b566c4f60d949d476ceb Jun 03 15:40:55.666113 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:55.666072 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98bd1db_934b_442e_8851_e8a60695aa8d.slice/crio-ab23d48dcd84207789df2630e49b3e0b99bab87b9f6d0caab1faf38eca832423 WatchSource:0}: Error finding container ab23d48dcd84207789df2630e49b3e0b99bab87b9f6d0caab1faf38eca832423: Status 404 returned error can't find the container with id ab23d48dcd84207789df2630e49b3e0b99bab87b9f6d0caab1faf38eca832423 Jun 03 15:40:55.667612 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:40:55.667589 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13e88df3_970c_472f_9f44_1a2102fa3695.slice/crio-2a8f68a7830079b55080663c297e8cc174fd21e244febd2ffe3ad98a87e0b1c1 WatchSource:0}: Error finding container 2a8f68a7830079b55080663c297e8cc174fd21e244febd2ffe3ad98a87e0b1c1: Status 404 returned error can't find the container with id 2a8f68a7830079b55080663c297e8cc174fd21e244febd2ffe3ad98a87e0b1c1 Jun 03 15:40:55.695012 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:55.694894 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnr4\" (UniqueName: \"kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4\") pod \"network-check-target-hm6g8\" (UID: \"43da5244-85e8-4634-8827-e268eb3df28b\") " pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:40:55.695110 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:55.695028 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:40:55.695110 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:55.695048 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:40:55.695110 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:55.695060 2578 projected.go:194] Error preparing data for projected volume kube-api-access-bfnr4 for pod openshift-network-diagnostics/network-check-target-hm6g8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:40:55.695110 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:55.695111 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4 podName:43da5244-85e8-4634-8827-e268eb3df28b nodeName:}" failed. No retries permitted until 2026-06-03 15:40:56.695093274 +0000 UTC m=+4.080160472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfnr4" (UniqueName: "kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4") pod "network-check-target-hm6g8" (UID: "43da5244-85e8-4634-8827-e268eb3df28b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:40:56.019558 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.019513 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 15:35:54 +0000 UTC" deadline="2028-02-23 09:06:10.649752468 +0000 UTC" Jun 03 15:40:56.019558 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.019554 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15113h25m14.630201387s" Jun 03 15:40:56.108291 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.107552 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:40:56.108291 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:56.107682 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:40:56.108291 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.108108 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:40:56.108291 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:56.108238 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:40:56.121269 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.121231 2578 generic.go:358] "Generic (PLEG): container finished" podID="94971eb58eb2e55ff58107d4a8f407cc" containerID="27c48305c99297146c293bc9121cd5f750b6562c68fa4c3e321517dc895ceba1" exitCode=0 Jun 03 15:40:56.121423 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.121321 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" event={"ID":"94971eb58eb2e55ff58107d4a8f407cc","Type":"ContainerDied","Data":"27c48305c99297146c293bc9121cd5f750b6562c68fa4c3e321517dc895ceba1"} Jun 03 15:40:56.130082 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.130052 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-129.ec2.internal" event={"ID":"43488e4976b4b669cc3a2c15df77a88a","Type":"ContainerStarted","Data":"5c8bcc1f587f866708ecd37a91154102784595a3d1ad88e229721196299848c5"} Jun 03 15:40:56.133440 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.133413 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jmlt" event={"ID":"a98bd1db-934b-442e-8851-e8a60695aa8d","Type":"ContainerStarted","Data":"ab23d48dcd84207789df2630e49b3e0b99bab87b9f6d0caab1faf38eca832423"} Jun 03 15:40:56.135124 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.135097 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v6zjk" event={"ID":"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e","Type":"ContainerStarted","Data":"125a1cfb3e11fbc9bc9765d8f45da129f7da3059bb288d09ed91724342df6214"} Jun 03 15:40:56.142929 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.142901 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" event={"ID":"a795c9e9-6262-491f-b6e3-db2a50aea8eb","Type":"ContainerStarted","Data":"2592ebe28322fb88b7bdf8d9e64f14fe49bfa32630a3e0a56e0d61c9feee5cc0"} Jun 03 15:40:56.151137 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.151108 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4hx4n" event={"ID":"e3c78079-acba-48d7-a32a-2dd02cb04d98","Type":"ContainerStarted","Data":"e69270cb6767d1c1f0474fc0cecf01d67d7fc8cc75fb720765a3ccb5c8fa20c4"} Jun 03 15:40:56.157320 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.155488 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4dfqm" event={"ID":"a239c904-33ce-4c96-88ce-c976799776e8","Type":"ContainerStarted","Data":"f82d3b7a4171f4f5b415cb3710896d95d52199e1fe1197626906666d18950234"} Jun 03 15:40:56.162831 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.162799 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" event={"ID":"13e88df3-970c-472f-9f44-1a2102fa3695","Type":"ContainerStarted","Data":"2a8f68a7830079b55080663c297e8cc174fd21e244febd2ffe3ad98a87e0b1c1"} Jun 03 15:40:56.165628 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.165595 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x74fd" event={"ID":"e5f1cad3-acab-45ec-8328-0161d1e9942a","Type":"ContainerStarted","Data":"9458054cb2ee6cc463df4d8130820d088387090cf4f4b566c4f60d949d476ceb"} Jun 03 15:40:56.167747 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.167711 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gh9z6" event={"ID":"22b6b156-cdf5-4696-b64c-fec58fcc28e5","Type":"ContainerStarted","Data":"0e15ae51525626a3fd0ed0fdb9d87a3b55d03d46d9ba1c10b7190924165b6889"} Jun 03 15:40:56.170191 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.170141 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" event={"ID":"f94fa647-de64-4615-8168-d3521878583e","Type":"ContainerStarted","Data":"f570471f3a473f84846b105a688d961961a24edc1675a01c3b250bc8118fcd3a"} Jun 03 15:40:56.610126 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.610035 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:40:56.610340 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:56.610246 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:40:56.610340 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:56.610313 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs podName:a9d8c3c0-82e0-48b5-909a-a7887d92baa9 nodeName:}" failed. No retries permitted until 2026-06-03 15:40:58.610293042 +0000 UTC m=+5.995360259 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs") pod "network-metrics-daemon-t7kx5" (UID: "a9d8c3c0-82e0-48b5-909a-a7887d92baa9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:40:56.710944 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.710906 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnr4\" (UniqueName: \"kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4\") pod \"network-check-target-hm6g8\" (UID: \"43da5244-85e8-4634-8827-e268eb3df28b\") " pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:40:56.711115 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:56.711068 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:40:56.711115 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:56.711086 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:40:56.711115 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:56.711099 2578 projected.go:194] Error preparing data for projected volume kube-api-access-bfnr4 for pod openshift-network-diagnostics/network-check-target-hm6g8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:40:56.711309 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:56.711158 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4 podName:43da5244-85e8-4634-8827-e268eb3df28b nodeName:}" failed. No retries permitted until 2026-06-03 15:40:58.711139257 +0000 UTC m=+6.096206456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfnr4" (UniqueName: "kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4") pod "network-check-target-hm6g8" (UID: "43da5244-85e8-4634-8827-e268eb3df28b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:40:56.815374 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:56.815167 2578 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 15:40:57.189817 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:57.189761 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" event={"ID":"94971eb58eb2e55ff58107d4a8f407cc","Type":"ContainerStarted","Data":"7332412e251d4fdcd6320b0c34469426476bd990bf64574e6525d4cadb3f152f"} Jun 03 15:40:57.203583 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:57.202761 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-129.ec2.internal" podStartSLOduration=3.202742678 podStartE2EDuration="3.202742678s" podCreationTimestamp="2026-06-03 15:40:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 15:40:57.202454262 +0000 UTC m=+4.587521485" watchObservedRunningTime="2026-06-03 15:40:57.202742678 +0000 UTC m=+4.587809897" Jun 03 15:40:57.203583 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:57.203186 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-129.ec2.internal" podStartSLOduration=3.203175994 podStartE2EDuration="3.203175994s" podCreationTimestamp="2026-06-03 15:40:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 15:40:56.151260652 +0000 UTC m=+3.536327873" watchObservedRunningTime="2026-06-03 15:40:57.203175994 +0000 UTC m=+4.588243215" Jun 03 15:40:58.107742 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:58.107187 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:40:58.107742 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:58.107340 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:40:58.107742 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:58.107393 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:40:58.107742 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:58.107514 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:40:58.626616 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:58.625958 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:40:58.626616 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:58.626176 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:40:58.626616 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:58.626258 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs podName:a9d8c3c0-82e0-48b5-909a-a7887d92baa9 nodeName:}" failed. No retries permitted until 2026-06-03 15:41:02.626239467 +0000 UTC m=+10.011306678 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs") pod "network-metrics-daemon-t7kx5" (UID: "a9d8c3c0-82e0-48b5-909a-a7887d92baa9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:40:58.727478 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:40:58.727234 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnr4\" (UniqueName: \"kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4\") pod \"network-check-target-hm6g8\" (UID: \"43da5244-85e8-4634-8827-e268eb3df28b\") " pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:40:58.727478 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:58.727438 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:40:58.727721 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:58.727503 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:40:58.727721 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:58.727530 2578 projected.go:194] Error preparing data for projected volume kube-api-access-bfnr4 for pod openshift-network-diagnostics/network-check-target-hm6g8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:40:58.727721 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:40:58.727591 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4 podName:43da5244-85e8-4634-8827-e268eb3df28b nodeName:}" failed. No retries permitted until 2026-06-03 15:41:02.727572719 +0000 UTC m=+10.112639928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfnr4" (UniqueName: "kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4") pod "network-check-target-hm6g8" (UID: "43da5244-85e8-4634-8827-e268eb3df28b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:41:00.107262 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:00.107229 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:00.107702 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:00.107278 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:00.107702 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:00.107360 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:00.107702 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:00.107438 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:02.108191 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:02.108156 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:02.108760 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:02.108160 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:02.108760 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:02.108317 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:02.108760 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:02.108458 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:02.663497 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:02.663244 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:02.663497 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:02.663395 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:41:02.663497 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:02.663470 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs podName:a9d8c3c0-82e0-48b5-909a-a7887d92baa9 nodeName:}" failed. No retries permitted until 2026-06-03 15:41:10.663448648 +0000 UTC m=+18.048515853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs") pod "network-metrics-daemon-t7kx5" (UID: "a9d8c3c0-82e0-48b5-909a-a7887d92baa9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:41:02.763806 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:02.763765 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnr4\" (UniqueName: \"kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4\") pod \"network-check-target-hm6g8\" (UID: \"43da5244-85e8-4634-8827-e268eb3df28b\") " pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:02.763985 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:02.763960 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:41:02.764045 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:02.763987 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:41:02.764045 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:02.764002 2578 projected.go:194] Error preparing data for projected volume kube-api-access-bfnr4 for pod openshift-network-diagnostics/network-check-target-hm6g8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:41:02.764153 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:02.764075 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4 podName:43da5244-85e8-4634-8827-e268eb3df28b nodeName:}" failed. No retries permitted until 2026-06-03 15:41:10.764053122 +0000 UTC m=+18.149120342 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfnr4" (UniqueName: "kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4") pod "network-check-target-hm6g8" (UID: "43da5244-85e8-4634-8827-e268eb3df28b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:41:04.107970 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:04.107872 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:04.107970 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:04.107918 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:04.108519 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:04.108000 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:04.108519 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:04.108478 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:06.107611 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:06.107573 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:06.107611 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:06.107621 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:06.108110 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:06.107721 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:06.108110 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:06.107856 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:08.107980 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:08.107946 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:08.108429 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:08.107946 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:08.108429 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:08.108077 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:08.108429 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:08.108116 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:10.107780 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:10.107745 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:10.108280 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:10.107866 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:10.108280 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:10.107931 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:10.108280 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:10.108060 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:10.719087 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:10.719052 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:10.719278 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:10.719254 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:41:10.719367 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:10.719340 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs podName:a9d8c3c0-82e0-48b5-909a-a7887d92baa9 nodeName:}" failed. No retries permitted until 2026-06-03 15:41:26.719321798 +0000 UTC m=+34.104388995 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs") pod "network-metrics-daemon-t7kx5" (UID: "a9d8c3c0-82e0-48b5-909a-a7887d92baa9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:41:10.819667 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:10.819625 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnr4\" (UniqueName: \"kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4\") pod \"network-check-target-hm6g8\" (UID: \"43da5244-85e8-4634-8827-e268eb3df28b\") " pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:10.819833 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:10.819784 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:41:10.819833 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:10.819810 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:41:10.819833 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:10.819826 2578 projected.go:194] Error preparing data for projected volume kube-api-access-bfnr4 for pod openshift-network-diagnostics/network-check-target-hm6g8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:41:10.819952 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:10.819895 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4 podName:43da5244-85e8-4634-8827-e268eb3df28b nodeName:}" failed. No retries permitted until 2026-06-03 15:41:26.819875557 +0000 UTC m=+34.204942757 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfnr4" (UniqueName: "kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4") pod "network-check-target-hm6g8" (UID: "43da5244-85e8-4634-8827-e268eb3df28b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:41:12.107775 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:12.107736 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:12.108251 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:12.107738 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:12.108251 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:12.107877 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:12.108251 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:12.107977 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:14.108191 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.107981 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:14.108738 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.107992 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:14.108738 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:14.108308 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:14.108738 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:14.108398 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:14.223354 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.223164 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" event={"ID":"a795c9e9-6262-491f-b6e3-db2a50aea8eb","Type":"ContainerStarted","Data":"66809600e8423237f1f6870655f7cdabb3cb6442a68c29f0595b95f431425a9e"} Jun 03 15:41:14.224644 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.224594 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4hx4n" event={"ID":"e3c78079-acba-48d7-a32a-2dd02cb04d98","Type":"ContainerStarted","Data":"8b3f9a05fdbc98524267b2061249ad261df323d5e73bf630d7c44f6498fda4de"} Jun 03 15:41:14.225923 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.225901 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4dfqm" event={"ID":"a239c904-33ce-4c96-88ce-c976799776e8","Type":"ContainerStarted","Data":"bd8ca731d2055d32460b80c606a613e1bb4d29bf05bbaf6e75d13b2e6bbb7764"} Jun 03 15:41:14.227973 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.227953 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 15:41:14.228305 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.228287 2578 generic.go:358] "Generic (PLEG): container finished" podID="13e88df3-970c-472f-9f44-1a2102fa3695" containerID="78c0364cb8a8a692929991ad69f53aaa87936dc64a31a75b2514a226e71f7e7d" exitCode=1 Jun 03 15:41:14.228385 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.228313 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" event={"ID":"13e88df3-970c-472f-9f44-1a2102fa3695","Type":"ContainerStarted","Data":"7c06fe8acbbe0490f1355285e302b39365a6913ae840ef33653885a5ee893408"} Jun 03 15:41:14.228385 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.228343 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" event={"ID":"13e88df3-970c-472f-9f44-1a2102fa3695","Type":"ContainerDied","Data":"78c0364cb8a8a692929991ad69f53aaa87936dc64a31a75b2514a226e71f7e7d"} Jun 03 15:41:14.228385 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.228360 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" event={"ID":"13e88df3-970c-472f-9f44-1a2102fa3695","Type":"ContainerStarted","Data":"9bef87a186c085df49a27f3fe82f434f50a6e2e43723c6f9d38e9e7795f86b4a"} Jun 03 15:41:14.229862 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.229841 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gh9z6" event={"ID":"22b6b156-cdf5-4696-b64c-fec58fcc28e5","Type":"ContainerStarted","Data":"1afd89d7483f0761842ce3382516f5c35583b4be8dece5a0ba521d7be8a3dc0f"} Jun 03 15:41:14.231357 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.231325 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" event={"ID":"f94fa647-de64-4615-8168-d3521878583e","Type":"ContainerStarted","Data":"54dcd88884d5ad767eb6d686a83e6cf2ac26ffe1265af1e59e32d2ec05366b62"} Jun 03 15:41:14.232644 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.232620 2578 generic.go:358] "Generic (PLEG): container finished" podID="a98bd1db-934b-442e-8851-e8a60695aa8d" containerID="9967c110a25616003377ff1736626eb2cbf672ef6bfabcc1e5784986f291cfb4" exitCode=0 Jun 03 15:41:14.232747 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.232688 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jmlt" event={"ID":"a98bd1db-934b-442e-8851-e8a60695aa8d","Type":"ContainerDied","Data":"9967c110a25616003377ff1736626eb2cbf672ef6bfabcc1e5784986f291cfb4"} Jun 03 15:41:14.234180 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.234157 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v6zjk" event={"ID":"b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e","Type":"ContainerStarted","Data":"928193f83f405bd34700d33ac230008e82623497724b63198ac53a95ad835448"} Jun 03 15:41:14.242673 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.242638 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p9b7b" podStartSLOduration=3.439166542 podStartE2EDuration="21.242627697s" podCreationTimestamp="2026-06-03 15:40:53 +0000 UTC" firstStartedPulling="2026-06-03 15:40:55.665314384 +0000 UTC m=+3.050381596" lastFinishedPulling="2026-06-03 15:41:13.468775548 +0000 UTC m=+20.853842751" observedRunningTime="2026-06-03 15:41:14.242323057 +0000 UTC m=+21.627390276" watchObservedRunningTime="2026-06-03 15:41:14.242627697 +0000 UTC m=+21.627694915" Jun 03 15:41:14.256980 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.256931 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gh9z6" podStartSLOduration=3.452158173 podStartE2EDuration="21.256917865s" podCreationTimestamp="2026-06-03 15:40:53 +0000 UTC" firstStartedPulling="2026-06-03 15:40:55.664010321 +0000 UTC m=+3.049077518" lastFinishedPulling="2026-06-03 15:41:13.468769999 +0000 UTC m=+20.853837210" observedRunningTime="2026-06-03 15:41:14.256500807 +0000 UTC m=+21.641568027" watchObservedRunningTime="2026-06-03 15:41:14.256917865 +0000 UTC m=+21.641985082" Jun 03 15:41:14.273686 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.273643 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-4hx4n" podStartSLOduration=3.475117497 podStartE2EDuration="21.273630126s" podCreationTimestamp="2026-06-03 15:40:53 +0000 UTC" firstStartedPulling="2026-06-03 15:40:55.670297813 +0000 UTC m=+3.055365017" lastFinishedPulling="2026-06-03 15:41:13.468810446 +0000 UTC m=+20.853877646" observedRunningTime="2026-06-03 15:41:14.272767298 +0000 UTC m=+21.657834516" watchObservedRunningTime="2026-06-03 15:41:14.273630126 +0000 UTC m=+21.658697344" Jun 03 15:41:14.292726 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.292665 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-v6zjk" podStartSLOduration=3.456295234 podStartE2EDuration="21.292631812s" podCreationTimestamp="2026-06-03 15:40:53 +0000 UTC" firstStartedPulling="2026-06-03 15:40:55.667267255 +0000 UTC m=+3.052334457" lastFinishedPulling="2026-06-03 15:41:13.503603817 +0000 UTC m=+20.888671035" observedRunningTime="2026-06-03 15:41:14.291877202 +0000 UTC m=+21.676944422" watchObservedRunningTime="2026-06-03 15:41:14.292631812 +0000 UTC m=+21.677699034" Jun 03 15:41:14.307224 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.307103 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4dfqm" podStartSLOduration=3.4981313529999998 podStartE2EDuration="21.307089361s" podCreationTimestamp="2026-06-03 15:40:53 +0000 UTC" firstStartedPulling="2026-06-03 15:40:55.659812637 +0000 UTC m=+3.044879839" lastFinishedPulling="2026-06-03 15:41:13.468770646 +0000 UTC m=+20.853837847" observedRunningTime="2026-06-03 15:41:14.306797215 +0000 UTC m=+21.691864435" watchObservedRunningTime="2026-06-03 15:41:14.307089361 +0000 UTC m=+21.692156581" Jun 03 15:41:14.913799 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.913716 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 03 15:41:14.995987 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.995939 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4dfqm" Jun 03 15:41:14.996735 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:14.996701 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4dfqm" Jun 03 15:41:15.044918 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:15.044816 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-03T15:41:14.913733348Z","UUID":"23e619b5-9ef4-434c-8982-21ec1fa4d9ac","Handler":null,"Name":"","Endpoint":""} Jun 03 15:41:15.046646 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:15.046625 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 03 15:41:15.046646 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:15.046652 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 03 15:41:15.237957 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:15.237906 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x74fd" event={"ID":"e5f1cad3-acab-45ec-8328-0161d1e9942a","Type":"ContainerStarted","Data":"eeac9094a9dc368cc7e4af434770d2eb84a27ff86fc7f766878d925d707a4bae"} Jun 03 15:41:15.240336 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:15.240307 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" event={"ID":"f94fa647-de64-4615-8168-d3521878583e","Type":"ContainerStarted","Data":"120f1b14b05943d1f2ee6dcc465e2a8b021bf5692c38cd6fba5d928afdccf73d"} Jun 03 15:41:15.242979 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:15.242962 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 15:41:15.243386 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:15.243310 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" event={"ID":"13e88df3-970c-472f-9f44-1a2102fa3695","Type":"ContainerStarted","Data":"5634f53f71b91d7c092a0c51b4e6a8c63c2c7885c5f58d50b52f0d15a495e167"} Jun 03 15:41:15.243386 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:15.243342 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" event={"ID":"13e88df3-970c-472f-9f44-1a2102fa3695","Type":"ContainerStarted","Data":"fdb800d531bec61b0c156cdc8c335da897befb6898ea21707d4f957f563bd25e"} Jun 03 15:41:15.243386 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:15.243361 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" event={"ID":"13e88df3-970c-472f-9f44-1a2102fa3695","Type":"ContainerStarted","Data":"1b97bc12e4c798b97eb943ad7af39f461723b918501b3abc9b349bf4bdac9d91"} Jun 03 15:41:15.244244 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:15.244076 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4dfqm" Jun 03 15:41:15.244338 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:15.244282 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4dfqm" Jun 03 15:41:15.268533 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:15.268485 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x74fd" podStartSLOduration=4.443673105 podStartE2EDuration="22.268472513s" podCreationTimestamp="2026-06-03 15:40:53 +0000 UTC" firstStartedPulling="2026-06-03 15:40:55.667857647 +0000 UTC m=+3.052924861" lastFinishedPulling="2026-06-03 15:41:13.492657068 +0000 UTC m=+20.877724269" observedRunningTime="2026-06-03 15:41:15.253413823 +0000 UTC m=+22.638481046" watchObservedRunningTime="2026-06-03 15:41:15.268472513 +0000 UTC m=+22.653539742" Jun 03 15:41:16.107452 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:16.107416 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:16.107679 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:16.107421 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:16.107679 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:16.107528 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:16.107679 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:16.107631 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:16.247220 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:16.247170 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" event={"ID":"f94fa647-de64-4615-8168-d3521878583e","Type":"ContainerStarted","Data":"20af80f9207bfe4f688131faea869c3ae64771356b4ca8a547ec573e2a54712c"} Jun 03 15:41:16.268767 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:16.268720 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jr2z8" podStartSLOduration=2.996647606 podStartE2EDuration="23.268707547s" podCreationTimestamp="2026-06-03 15:40:53 +0000 UTC" firstStartedPulling="2026-06-03 15:40:55.660172151 +0000 UTC m=+3.045239348" lastFinishedPulling="2026-06-03 15:41:15.932232068 +0000 UTC m=+23.317299289" observedRunningTime="2026-06-03 15:41:16.268347839 +0000 UTC m=+23.653415063" watchObservedRunningTime="2026-06-03 15:41:16.268707547 +0000 UTC m=+23.653774796" Jun 03 15:41:18.107680 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:18.107479 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:18.108144 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:18.107487 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:18.108144 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:18.107772 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:18.108144 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:18.107863 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:19.254021 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:19.253988 2578 generic.go:358] "Generic (PLEG): container finished" podID="a98bd1db-934b-442e-8851-e8a60695aa8d" containerID="d117c0589b27af15f37d88641d185de57b598885f428d876121fa7ee19946b83" exitCode=0 Jun 03 15:41:19.254674 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:19.254072 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jmlt" event={"ID":"a98bd1db-934b-442e-8851-e8a60695aa8d","Type":"ContainerDied","Data":"d117c0589b27af15f37d88641d185de57b598885f428d876121fa7ee19946b83"} Jun 03 15:41:19.256745 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:19.256725 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 15:41:19.257072 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:19.257051 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" event={"ID":"13e88df3-970c-472f-9f44-1a2102fa3695","Type":"ContainerStarted","Data":"ffdd5a25bf6e8c596c1f97806acdd04eadfe483704ae19d747c7f176ae05b534"} Jun 03 15:41:20.108216 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:20.108165 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:20.108377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:20.108225 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:20.108377 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:20.108305 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:20.108446 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:20.108418 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:21.262006 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:21.261804 2578 generic.go:358] "Generic (PLEG): container finished" podID="a98bd1db-934b-442e-8851-e8a60695aa8d" containerID="d3952888eeef5d918fc0dac5d4eeed569c1568bda981751e25d7c83f40731a3e" exitCode=0 Jun 03 15:41:21.262590 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:21.261895 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jmlt" event={"ID":"a98bd1db-934b-442e-8851-e8a60695aa8d","Type":"ContainerDied","Data":"d3952888eeef5d918fc0dac5d4eeed569c1568bda981751e25d7c83f40731a3e"} Jun 03 15:41:21.265077 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:21.265059 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 15:41:21.265419 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:21.265396 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" event={"ID":"13e88df3-970c-472f-9f44-1a2102fa3695","Type":"ContainerStarted","Data":"5d9057041144d93fd35cd9a65c2da5b752631bc812499ef9902091cf9de94487"} Jun 03 15:41:21.265695 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:21.265681 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:41:21.265744 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:21.265704 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:41:21.265744 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:21.265718 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:41:21.265860 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:21.265846 2578 scope.go:117] "RemoveContainer" containerID="78c0364cb8a8a692929991ad69f53aaa87936dc64a31a75b2514a226e71f7e7d" Jun 03 15:41:21.282303 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:21.282276 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:41:21.282398 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:21.282355 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:41:22.107904 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:22.107869 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:22.108063 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:22.107923 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:22.108063 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:22.108025 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:22.108213 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:22.108147 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:22.273005 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:22.272985 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 15:41:22.273349 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:22.273264 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" event={"ID":"13e88df3-970c-472f-9f44-1a2102fa3695","Type":"ContainerStarted","Data":"1bf0345ba92864bc7c72d85e7b97d8505b42b548ca85beb62a43f5e914f87ccd"} Jun 03 15:41:22.313011 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:22.312948 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" podStartSLOduration=11.133180144 podStartE2EDuration="29.312930873s" podCreationTimestamp="2026-06-03 15:40:53 +0000 UTC" firstStartedPulling="2026-06-03 15:40:55.670089849 +0000 UTC m=+3.055157045" lastFinishedPulling="2026-06-03 15:41:13.849840569 +0000 UTC m=+21.234907774" observedRunningTime="2026-06-03 15:41:22.307523374 +0000 UTC m=+29.692590587" watchObservedRunningTime="2026-06-03 15:41:22.312930873 +0000 UTC m=+29.697998092" Jun 03 15:41:22.608147 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:22.608106 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t7kx5"] Jun 03 15:41:22.608518 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:22.608288 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:22.608518 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:22.608413 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:22.611468 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:22.611229 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hm6g8"] Jun 03 15:41:22.611468 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:22.611342 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:22.611468 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:22.611428 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:23.276956 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:23.276770 2578 generic.go:358] "Generic (PLEG): container finished" podID="a98bd1db-934b-442e-8851-e8a60695aa8d" containerID="b29d3a8f87378e3d03e8d6d2a7d702d12ce76b3266b90f654f8561603b23440c" exitCode=0 Jun 03 15:41:23.277362 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:23.276850 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jmlt" event={"ID":"a98bd1db-934b-442e-8851-e8a60695aa8d","Type":"ContainerDied","Data":"b29d3a8f87378e3d03e8d6d2a7d702d12ce76b3266b90f654f8561603b23440c"} Jun 03 15:41:24.107930 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:24.107890 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:24.108134 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:24.107890 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:24.108134 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:24.108045 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:24.108245 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:24.108171 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:26.107994 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:26.107958 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:26.108412 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:26.108072 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hm6g8" podUID="43da5244-85e8-4634-8827-e268eb3df28b" Jun 03 15:41:26.108412 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:26.108136 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:26.108412 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:26.108279 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:41:26.739939 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:26.739903 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:26.740168 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:26.740042 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:41:26.740168 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:26.740107 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs podName:a9d8c3c0-82e0-48b5-909a-a7887d92baa9 nodeName:}" failed. No retries permitted until 2026-06-03 15:41:58.740089408 +0000 UTC m=+66.125156604 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs") pod "network-metrics-daemon-t7kx5" (UID: "a9d8c3c0-82e0-48b5-909a-a7887d92baa9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:41:26.840269 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:26.840236 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnr4\" (UniqueName: \"kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4\") pod \"network-check-target-hm6g8\" (UID: \"43da5244-85e8-4634-8827-e268eb3df28b\") " pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:26.840509 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:26.840375 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:41:26.840509 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:26.840406 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:41:26.840509 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:26.840420 2578 projected.go:194] Error preparing data for projected volume kube-api-access-bfnr4 for pod openshift-network-diagnostics/network-check-target-hm6g8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:41:26.840509 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:26.840486 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4 podName:43da5244-85e8-4634-8827-e268eb3df28b nodeName:}" failed. No retries permitted until 2026-06-03 15:41:58.840467416 +0000 UTC m=+66.225534615 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfnr4" (UniqueName: "kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4") pod "network-check-target-hm6g8" (UID: "43da5244-85e8-4634-8827-e268eb3df28b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:41:27.488822 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.488790 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-129.ec2.internal" event="NodeReady" Jun 03 15:41:27.489312 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.488948 2578 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 03 15:41:27.538606 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.538575 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n4lvz"] Jun 03 15:41:27.565704 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.565651 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-96wm8"] Jun 03 15:41:27.566118 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.565717 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:27.568479 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.568453 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 03 15:41:27.568961 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.568945 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 03 15:41:27.569110 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.569094 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2z9xs\"" Jun 03 15:41:27.579538 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.579519 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n4lvz"] Jun 03 15:41:27.579538 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.579542 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-96wm8"] Jun 03 15:41:27.579871 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.579627 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:41:27.582105 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.582086 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 03 15:41:27.582239 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.582145 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-f2mct\"" Jun 03 15:41:27.582511 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.582494 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 03 15:41:27.582802 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.582786 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 03 15:41:27.645514 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.645477 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-config-volume\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:27.645514 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.645519 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:27.645756 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.645548 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phdlf\" (UniqueName: \"kubernetes.io/projected/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-kube-api-access-phdlf\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:27.645756 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.645618 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-tmp-dir\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:27.746481 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.746409 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-config-volume\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:27.746481 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.746441 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:27.746481 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.746467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:41:27.746682 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.746490 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-phdlf\" (UniqueName: \"kubernetes.io/projected/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-kube-api-access-phdlf\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:27.746682 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:27.746585 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:41:27.746682 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:27.746659 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls podName:6cb3b438-55f9-4194-ba2a-e91afec5a1e4 nodeName:}" failed. No retries permitted until 2026-06-03 15:41:28.246639042 +0000 UTC m=+35.631706241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls") pod "dns-default-n4lvz" (UID: "6cb3b438-55f9-4194-ba2a-e91afec5a1e4") : secret "dns-default-metrics-tls" not found Jun 03 15:41:27.746791 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.746683 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-tmp-dir\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:27.746791 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.746736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxdn8\" (UniqueName: \"kubernetes.io/projected/5ca43b94-b17e-4492-a995-30971a21f97b-kube-api-access-gxdn8\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:41:27.746978 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.746962 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-tmp-dir\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:27.747018 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.746985 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-config-volume\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:27.759578 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.759554 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phdlf\" (UniqueName: \"kubernetes.io/projected/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-kube-api-access-phdlf\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:27.847595 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.847558 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gxdn8\" (UniqueName: \"kubernetes.io/projected/5ca43b94-b17e-4492-a995-30971a21f97b-kube-api-access-gxdn8\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:41:27.847766 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.847621 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:41:27.847832 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:27.847779 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:41:27.847888 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:27.847849 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert podName:5ca43b94-b17e-4492-a995-30971a21f97b nodeName:}" failed. No retries permitted until 2026-06-03 15:41:28.347827375 +0000 UTC m=+35.732894575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert") pod "ingress-canary-96wm8" (UID: "5ca43b94-b17e-4492-a995-30971a21f97b") : secret "canary-serving-cert" not found Jun 03 15:41:27.856718 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:27.856692 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxdn8\" (UniqueName: \"kubernetes.io/projected/5ca43b94-b17e-4492-a995-30971a21f97b-kube-api-access-gxdn8\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:41:28.108158 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:28.108068 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:28.108337 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:28.108285 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:28.111128 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:28.111107 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 15:41:28.111265 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:28.111140 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 15:41:28.111265 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:28.111246 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7qpdh\"" Jun 03 15:41:28.111376 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:28.111275 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 15:41:28.111434 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:28.111394 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q8ntl\"" Jun 03 15:41:28.251125 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:28.251092 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:28.251339 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:28.251314 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:41:28.251403 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:28.251391 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls podName:6cb3b438-55f9-4194-ba2a-e91afec5a1e4 nodeName:}" failed. No retries permitted until 2026-06-03 15:41:29.25136332 +0000 UTC m=+36.636430517 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls") pod "dns-default-n4lvz" (UID: "6cb3b438-55f9-4194-ba2a-e91afec5a1e4") : secret "dns-default-metrics-tls" not found Jun 03 15:41:28.351519 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:28.351477 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:41:28.351677 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:28.351651 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:41:28.351740 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:28.351729 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert podName:5ca43b94-b17e-4492-a995-30971a21f97b nodeName:}" failed. No retries permitted until 2026-06-03 15:41:29.351708792 +0000 UTC m=+36.736775991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert") pod "ingress-canary-96wm8" (UID: "5ca43b94-b17e-4492-a995-30971a21f97b") : secret "canary-serving-cert" not found Jun 03 15:41:29.258905 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:29.258864 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:29.259445 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:29.259026 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:41:29.259445 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:29.259103 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls podName:6cb3b438-55f9-4194-ba2a-e91afec5a1e4 nodeName:}" failed. No retries permitted until 2026-06-03 15:41:31.259086662 +0000 UTC m=+38.644153858 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls") pod "dns-default-n4lvz" (UID: "6cb3b438-55f9-4194-ba2a-e91afec5a1e4") : secret "dns-default-metrics-tls" not found Jun 03 15:41:29.359910 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:29.359870 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:41:29.360100 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:29.360046 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:41:29.360173 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:29.360131 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert podName:5ca43b94-b17e-4492-a995-30971a21f97b nodeName:}" failed. No retries permitted until 2026-06-03 15:41:31.360103489 +0000 UTC m=+38.745170692 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert") pod "ingress-canary-96wm8" (UID: "5ca43b94-b17e-4492-a995-30971a21f97b") : secret "canary-serving-cert" not found Jun 03 15:41:30.294180 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:30.294093 2578 generic.go:358] "Generic (PLEG): container finished" podID="a98bd1db-934b-442e-8851-e8a60695aa8d" containerID="5f0f186a59540459d6e1ec1974f38e59905376e6a3997c23e802550221c0ac37" exitCode=0 Jun 03 15:41:30.294180 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:30.294147 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jmlt" event={"ID":"a98bd1db-934b-442e-8851-e8a60695aa8d","Type":"ContainerDied","Data":"5f0f186a59540459d6e1ec1974f38e59905376e6a3997c23e802550221c0ac37"} Jun 03 15:41:31.275284 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:31.275253 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:31.275479 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:31.275375 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:41:31.275479 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:31.275430 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls podName:6cb3b438-55f9-4194-ba2a-e91afec5a1e4 nodeName:}" failed. No retries permitted until 2026-06-03 15:41:35.275415451 +0000 UTC m=+42.660482647 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls") pod "dns-default-n4lvz" (UID: "6cb3b438-55f9-4194-ba2a-e91afec5a1e4") : secret "dns-default-metrics-tls" not found Jun 03 15:41:31.298776 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:31.298739 2578 generic.go:358] "Generic (PLEG): container finished" podID="a98bd1db-934b-442e-8851-e8a60695aa8d" containerID="4bd5ed5d192684cfa5c93c74535ee921fcbae0c076cc35ab0f1f64140fea8c26" exitCode=0 Jun 03 15:41:31.299135 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:31.298792 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jmlt" event={"ID":"a98bd1db-934b-442e-8851-e8a60695aa8d","Type":"ContainerDied","Data":"4bd5ed5d192684cfa5c93c74535ee921fcbae0c076cc35ab0f1f64140fea8c26"} Jun 03 15:41:31.376310 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:31.376279 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:41:31.376431 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:31.376391 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:41:31.376469 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:31.376441 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert podName:5ca43b94-b17e-4492-a995-30971a21f97b nodeName:}" failed. No retries permitted until 2026-06-03 15:41:35.376427502 +0000 UTC m=+42.761494700 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert") pod "ingress-canary-96wm8" (UID: "5ca43b94-b17e-4492-a995-30971a21f97b") : secret "canary-serving-cert" not found Jun 03 15:41:32.303290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.303251 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4jmlt" event={"ID":"a98bd1db-934b-442e-8851-e8a60695aa8d","Type":"ContainerStarted","Data":"c9c988af23e8d6d338c9daaec788c53063998b399c26c003f57e24635d7581cc"} Jun 03 15:41:32.326946 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.326894 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4jmlt" podStartSLOduration=5.003784079 podStartE2EDuration="39.326880932s" podCreationTimestamp="2026-06-03 15:40:53 +0000 UTC" firstStartedPulling="2026-06-03 15:40:55.668784609 +0000 UTC m=+3.053851807" lastFinishedPulling="2026-06-03 15:41:29.991881458 +0000 UTC m=+37.376948660" observedRunningTime="2026-06-03 15:41:32.325934723 +0000 UTC m=+39.711001980" watchObservedRunningTime="2026-06-03 15:41:32.326880932 +0000 UTC m=+39.711948152" Jun 03 15:41:32.656995 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.656949 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x"] Jun 03 15:41:32.661058 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.661042 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn"] Jun 03 15:41:32.661232 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.661215 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:32.663571 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.663543 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 03 15:41:32.663682 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.663591 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 03 15:41:32.663682 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.663602 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 03 15:41:32.663845 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.663830 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 03 15:41:32.665284 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.665265 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.667302 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.667283 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 03 15:41:32.667520 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.667503 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 03 15:41:32.667567 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.667550 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 03 15:41:32.667651 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.667616 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 03 15:41:32.671518 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.671498 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x"] Jun 03 15:41:32.672423 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.672407 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn"] Jun 03 15:41:32.787521 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.787478 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f967aa1-5577-4dc3-b7dc-7229afe3f3b1-tmp\") pod \"klusterlet-addon-workmgr-575c4bdffd-62z5x\" (UID: \"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:32.787521 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.787522 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlbjt\" (UniqueName: \"kubernetes.io/projected/1f6a6e61-a7cf-4552-8859-3896315b5774-kube-api-access-dlbjt\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.787722 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.787589 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1f6a6e61-a7cf-4552-8859-3896315b5774-ca\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.787722 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.787630 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1f6a6e61-a7cf-4552-8859-3896315b5774-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.787722 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.787653 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7f967aa1-5577-4dc3-b7dc-7229afe3f3b1-klusterlet-config\") pod \"klusterlet-addon-workmgr-575c4bdffd-62z5x\" (UID: \"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:32.787722 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.787669 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjp28\" (UniqueName: \"kubernetes.io/projected/7f967aa1-5577-4dc3-b7dc-7229afe3f3b1-kube-api-access-cjp28\") pod \"klusterlet-addon-workmgr-575c4bdffd-62z5x\" (UID: \"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:32.787722 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.787692 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1f6a6e61-a7cf-4552-8859-3896315b5774-hub\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.787722 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.787715 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/1f6a6e61-a7cf-4552-8859-3896315b5774-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.787925 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.787738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1f6a6e61-a7cf-4552-8859-3896315b5774-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.888810 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.888779 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1f6a6e61-a7cf-4552-8859-3896315b5774-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.888810 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.888813 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7f967aa1-5577-4dc3-b7dc-7229afe3f3b1-klusterlet-config\") pod \"klusterlet-addon-workmgr-575c4bdffd-62z5x\" (UID: \"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:32.889034 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.888834 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cjp28\" (UniqueName: \"kubernetes.io/projected/7f967aa1-5577-4dc3-b7dc-7229afe3f3b1-kube-api-access-cjp28\") pod \"klusterlet-addon-workmgr-575c4bdffd-62z5x\" (UID: \"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:32.889034 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.888870 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1f6a6e61-a7cf-4552-8859-3896315b5774-hub\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.889034 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.888896 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/1f6a6e61-a7cf-4552-8859-3896315b5774-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.889034 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.888920 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1f6a6e61-a7cf-4552-8859-3896315b5774-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.889262 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.889177 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f967aa1-5577-4dc3-b7dc-7229afe3f3b1-tmp\") pod \"klusterlet-addon-workmgr-575c4bdffd-62z5x\" (UID: \"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:32.889262 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.889240 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dlbjt\" (UniqueName: \"kubernetes.io/projected/1f6a6e61-a7cf-4552-8859-3896315b5774-kube-api-access-dlbjt\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.889370 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.889266 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1f6a6e61-a7cf-4552-8859-3896315b5774-ca\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.889694 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.889667 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f967aa1-5577-4dc3-b7dc-7229afe3f3b1-tmp\") pod \"klusterlet-addon-workmgr-575c4bdffd-62z5x\" (UID: \"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:32.889828 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.889779 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/1f6a6e61-a7cf-4552-8859-3896315b5774-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.891652 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.891627 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1f6a6e61-a7cf-4552-8859-3896315b5774-ca\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.891851 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.891830 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1f6a6e61-a7cf-4552-8859-3896315b5774-hub\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.891890 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.891871 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7f967aa1-5577-4dc3-b7dc-7229afe3f3b1-klusterlet-config\") pod \"klusterlet-addon-workmgr-575c4bdffd-62z5x\" (UID: \"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:32.891991 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.891974 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1f6a6e61-a7cf-4552-8859-3896315b5774-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.892282 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.892264 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1f6a6e61-a7cf-4552-8859-3896315b5774-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.897738 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.897717 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjp28\" (UniqueName: \"kubernetes.io/projected/7f967aa1-5577-4dc3-b7dc-7229afe3f3b1-kube-api-access-cjp28\") pod \"klusterlet-addon-workmgr-575c4bdffd-62z5x\" (UID: \"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:32.898030 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.898005 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlbjt\" (UniqueName: \"kubernetes.io/projected/1f6a6e61-a7cf-4552-8859-3896315b5774-kube-api-access-dlbjt\") pod \"cluster-proxy-proxy-agent-69c9c66c-lp4pn\" (UID: \"1f6a6e61-a7cf-4552-8859-3896315b5774\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:32.971571 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.971535 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:32.987422 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:32.987392 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:41:33.114918 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:33.114889 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x"] Jun 03 15:41:33.118249 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:41:33.118184 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f967aa1_5577_4dc3_b7dc_7229afe3f3b1.slice/crio-dc94c19516c8b3b621ebefbae5b63eea16b39309df0a5af21408940c578c1b2d WatchSource:0}: Error finding container dc94c19516c8b3b621ebefbae5b63eea16b39309df0a5af21408940c578c1b2d: Status 404 returned error can't find the container with id dc94c19516c8b3b621ebefbae5b63eea16b39309df0a5af21408940c578c1b2d Jun 03 15:41:33.134632 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:33.134606 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn"] Jun 03 15:41:33.137441 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:41:33.137418 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f6a6e61_a7cf_4552_8859_3896315b5774.slice/crio-258f4a2ec9addb7f618b4106406b4527dffbe38e6cfd830d61b2382ee2e64f49 WatchSource:0}: Error finding container 258f4a2ec9addb7f618b4106406b4527dffbe38e6cfd830d61b2382ee2e64f49: Status 404 returned error can't find the container with id 258f4a2ec9addb7f618b4106406b4527dffbe38e6cfd830d61b2382ee2e64f49 Jun 03 15:41:33.305667 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:33.305580 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" event={"ID":"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1","Type":"ContainerStarted","Data":"dc94c19516c8b3b621ebefbae5b63eea16b39309df0a5af21408940c578c1b2d"} Jun 03 15:41:33.306561 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:33.306537 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" event={"ID":"1f6a6e61-a7cf-4552-8859-3896315b5774","Type":"ContainerStarted","Data":"258f4a2ec9addb7f618b4106406b4527dffbe38e6cfd830d61b2382ee2e64f49"} Jun 03 15:41:35.311068 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:35.311032 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:35.311536 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:35.311172 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:41:35.311536 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:35.311254 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls podName:6cb3b438-55f9-4194-ba2a-e91afec5a1e4 nodeName:}" failed. No retries permitted until 2026-06-03 15:41:43.311233651 +0000 UTC m=+50.696300851 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls") pod "dns-default-n4lvz" (UID: "6cb3b438-55f9-4194-ba2a-e91afec5a1e4") : secret "dns-default-metrics-tls" not found Jun 03 15:41:35.412165 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:35.412124 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:41:35.412368 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:35.412314 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:41:35.412368 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:35.412365 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert podName:5ca43b94-b17e-4492-a995-30971a21f97b nodeName:}" failed. No retries permitted until 2026-06-03 15:41:43.412349767 +0000 UTC m=+50.797416964 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert") pod "ingress-canary-96wm8" (UID: "5ca43b94-b17e-4492-a995-30971a21f97b") : secret "canary-serving-cert" not found Jun 03 15:41:39.321468 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:39.321428 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" event={"ID":"1f6a6e61-a7cf-4552-8859-3896315b5774","Type":"ContainerStarted","Data":"2c4c5b8e0b1e3658c23ef390c4523a82f987958857312892569638a207d07b3f"} Jun 03 15:41:39.322808 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:39.322776 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" event={"ID":"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1","Type":"ContainerStarted","Data":"21c51700f7a8d0873c8c29b7297ed473f1e935b0d305ac39aef71e75774f11b0"} Jun 03 15:41:39.323036 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:39.323016 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:39.324748 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:39.324722 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:41:39.346620 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:39.346416 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" podStartSLOduration=1.9754239660000001 podStartE2EDuration="7.346401623s" podCreationTimestamp="2026-06-03 15:41:32 +0000 UTC" firstStartedPulling="2026-06-03 15:41:33.120693159 +0000 UTC m=+40.505760359" lastFinishedPulling="2026-06-03 15:41:38.491670819 +0000 UTC m=+45.876738016" observedRunningTime="2026-06-03 15:41:39.346271754 +0000 UTC m=+46.731338986" watchObservedRunningTime="2026-06-03 15:41:39.346401623 +0000 UTC m=+46.731468843" Jun 03 15:41:41.328344 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:41.328318 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" event={"ID":"1f6a6e61-a7cf-4552-8859-3896315b5774","Type":"ContainerStarted","Data":"4a42d15a0378f00c2469b592e51ddc80a62e2f09cb900d8ce9c361b287b95d7b"} Jun 03 15:41:42.332403 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:42.332334 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" event={"ID":"1f6a6e61-a7cf-4552-8859-3896315b5774","Type":"ContainerStarted","Data":"3605b0a0ed742ecbc517746f0d3de74d2048fa3098efc17f838fdae3597c868f"} Jun 03 15:41:42.356564 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:42.356379 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" podStartSLOduration=2.405319822 podStartE2EDuration="10.356363942s" podCreationTimestamp="2026-06-03 15:41:32 +0000 UTC" firstStartedPulling="2026-06-03 15:41:33.138986424 +0000 UTC m=+40.524053627" lastFinishedPulling="2026-06-03 15:41:41.090030534 +0000 UTC m=+48.475097747" observedRunningTime="2026-06-03 15:41:42.355447299 +0000 UTC m=+49.740514518" watchObservedRunningTime="2026-06-03 15:41:42.356363942 +0000 UTC m=+49.741431160" Jun 03 15:41:43.366267 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:43.366230 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:43.366769 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:43.366368 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:41:43.366769 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:43.366431 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls podName:6cb3b438-55f9-4194-ba2a-e91afec5a1e4 nodeName:}" failed. No retries permitted until 2026-06-03 15:41:59.366410961 +0000 UTC m=+66.751478178 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls") pod "dns-default-n4lvz" (UID: "6cb3b438-55f9-4194-ba2a-e91afec5a1e4") : secret "dns-default-metrics-tls" not found Jun 03 15:41:43.467332 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:43.467299 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:41:43.467516 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:43.467442 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:41:43.467557 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:43.467520 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert podName:5ca43b94-b17e-4492-a995-30971a21f97b nodeName:}" failed. No retries permitted until 2026-06-03 15:41:59.46750392 +0000 UTC m=+66.852571120 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert") pod "ingress-canary-96wm8" (UID: "5ca43b94-b17e-4492-a995-30971a21f97b") : secret "canary-serving-cert" not found Jun 03 15:41:53.287822 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:53.287787 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kdw5d" Jun 03 15:41:58.774458 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:58.774400 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:41:58.795221 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:58.791470 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 15:41:58.795487 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:58.795463 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 03 15:41:58.795668 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:58.795655 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs podName:a9d8c3c0-82e0-48b5-909a-a7887d92baa9 nodeName:}" failed. No retries permitted until 2026-06-03 15:43:02.795632522 +0000 UTC m=+130.180699725 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs") pod "network-metrics-daemon-t7kx5" (UID: "a9d8c3c0-82e0-48b5-909a-a7887d92baa9") : secret "metrics-daemon-secret" not found Jun 03 15:41:58.874760 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:58.874723 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnr4\" (UniqueName: \"kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4\") pod \"network-check-target-hm6g8\" (UID: \"43da5244-85e8-4634-8827-e268eb3df28b\") " pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:58.877837 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:58.877818 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 15:41:58.888210 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:58.888179 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 15:41:58.898584 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:58.898564 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfnr4\" (UniqueName: \"kubernetes.io/projected/43da5244-85e8-4634-8827-e268eb3df28b-kube-api-access-bfnr4\") pod \"network-check-target-hm6g8\" (UID: \"43da5244-85e8-4634-8827-e268eb3df28b\") " pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:59.025113 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:59.025030 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7qpdh\"" Jun 03 15:41:59.030592 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:59.030572 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:41:59.149981 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:59.149944 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hm6g8"] Jun 03 15:41:59.152669 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:41:59.152635 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43da5244_85e8_4634_8827_e268eb3df28b.slice/crio-721216301cf5449a4e012b29f936e276b710eefcda4ff9a0366fe2d761d6be80 WatchSource:0}: Error finding container 721216301cf5449a4e012b29f936e276b710eefcda4ff9a0366fe2d761d6be80: Status 404 returned error can't find the container with id 721216301cf5449a4e012b29f936e276b710eefcda4ff9a0366fe2d761d6be80 Jun 03 15:41:59.366096 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:59.366006 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hm6g8" event={"ID":"43da5244-85e8-4634-8827-e268eb3df28b","Type":"ContainerStarted","Data":"721216301cf5449a4e012b29f936e276b710eefcda4ff9a0366fe2d761d6be80"} Jun 03 15:41:59.378457 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:59.378427 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:41:59.378584 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:59.378568 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:41:59.378646 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:59.378637 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls podName:6cb3b438-55f9-4194-ba2a-e91afec5a1e4 nodeName:}" failed. No retries permitted until 2026-06-03 15:42:31.378621066 +0000 UTC m=+98.763688274 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls") pod "dns-default-n4lvz" (UID: "6cb3b438-55f9-4194-ba2a-e91afec5a1e4") : secret "dns-default-metrics-tls" not found Jun 03 15:41:59.479607 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:41:59.479569 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:41:59.479760 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:59.479732 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:41:59.479823 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:41:59.479812 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert podName:5ca43b94-b17e-4492-a995-30971a21f97b nodeName:}" failed. No retries permitted until 2026-06-03 15:42:31.479789171 +0000 UTC m=+98.864856372 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert") pod "ingress-canary-96wm8" (UID: "5ca43b94-b17e-4492-a995-30971a21f97b") : secret "canary-serving-cert" not found Jun 03 15:42:03.379923 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:42:03.379882 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hm6g8" event={"ID":"43da5244-85e8-4634-8827-e268eb3df28b","Type":"ContainerStarted","Data":"f970015fd5a986ac6665f7fdbfd8936bef1c3fa47ed82ed4fe53e782ce752785"} Jun 03 15:42:03.380381 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:42:03.380058 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:42:03.396921 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:42:03.396877 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hm6g8" podStartSLOduration=66.919918279 podStartE2EDuration="1m10.396865438s" podCreationTimestamp="2026-06-03 15:40:53 +0000 UTC" firstStartedPulling="2026-06-03 15:41:59.154652694 +0000 UTC m=+66.539719894" lastFinishedPulling="2026-06-03 15:42:02.631599851 +0000 UTC m=+70.016667053" observedRunningTime="2026-06-03 15:42:03.396710141 +0000 UTC m=+70.781777361" watchObservedRunningTime="2026-06-03 15:42:03.396865438 +0000 UTC m=+70.781932651" Jun 03 15:42:31.401546 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:42:31.401500 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:42:31.402010 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:42:31.401664 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:42:31.402010 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:42:31.401747 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls podName:6cb3b438-55f9-4194-ba2a-e91afec5a1e4 nodeName:}" failed. No retries permitted until 2026-06-03 15:43:35.401729642 +0000 UTC m=+162.786796839 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls") pod "dns-default-n4lvz" (UID: "6cb3b438-55f9-4194-ba2a-e91afec5a1e4") : secret "dns-default-metrics-tls" not found Jun 03 15:42:31.502268 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:42:31.502223 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:42:31.502426 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:42:31.502354 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:42:31.502472 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:42:31.502428 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert podName:5ca43b94-b17e-4492-a995-30971a21f97b nodeName:}" failed. No retries permitted until 2026-06-03 15:43:35.502412249 +0000 UTC m=+162.887479449 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert") pod "ingress-canary-96wm8" (UID: "5ca43b94-b17e-4492-a995-30971a21f97b") : secret "canary-serving-cert" not found Jun 03 15:42:34.385546 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:42:34.385513 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hm6g8" Jun 03 15:43:02.814778 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:02.814704 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:43:02.815307 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:43:02.814857 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 03 15:43:02.815307 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:43:02.814938 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs podName:a9d8c3c0-82e0-48b5-909a-a7887d92baa9 nodeName:}" failed. No retries permitted until 2026-06-03 15:45:04.814921793 +0000 UTC m=+252.199988995 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs") pod "network-metrics-daemon-t7kx5" (UID: "a9d8c3c0-82e0-48b5-909a-a7887d92baa9") : secret "metrics-daemon-secret" not found Jun 03 15:43:06.562795 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:06.562764 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4hx4n_e3c78079-acba-48d7-a32a-2dd02cb04d98/dns-node-resolver/0.log" Jun 03 15:43:07.552765 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:07.552736 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gh9z6_22b6b156-cdf5-4696-b64c-fec58fcc28e5/node-ca/0.log" Jun 03 15:43:30.576723 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:43:30.576682 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-n4lvz" podUID="6cb3b438-55f9-4194-ba2a-e91afec5a1e4" Jun 03 15:43:30.589842 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:43:30.589811 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-96wm8" podUID="5ca43b94-b17e-4492-a995-30971a21f97b" Jun 03 15:43:31.125423 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:43:31.125390 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-t7kx5" podUID="a9d8c3c0-82e0-48b5-909a-a7887d92baa9" Jun 03 15:43:31.578393 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.578364 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n4lvz" Jun 03 15:43:31.606521 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.606493 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c65b8dc4b-rjdvs"] Jun 03 15:43:31.609391 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.609373 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-rjdvs" Jun 03 15:43:31.611978 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.611957 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmfn7\" (UniqueName: \"kubernetes.io/projected/aafc4657-83d5-4539-96fb-abcf974a2180-kube-api-access-xmfn7\") pod \"downloads-c65b8dc4b-rjdvs\" (UID: \"aafc4657-83d5-4539-96fb-abcf974a2180\") " pod="openshift-console/downloads-c65b8dc4b-rjdvs" Jun 03 15:43:31.612269 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.612247 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 03 15:43:31.612395 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.612307 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 03 15:43:31.612395 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.612309 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-c2v6j\"" Jun 03 15:43:31.623671 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.623650 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-rjdvs"] Jun 03 15:43:31.711610 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.711578 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-58fbcd44d-rw7xq"] Jun 03 15:43:31.712404 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.712383 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xmfn7\" (UniqueName: \"kubernetes.io/projected/aafc4657-83d5-4539-96fb-abcf974a2180-kube-api-access-xmfn7\") pod \"downloads-c65b8dc4b-rjdvs\" (UID: \"aafc4657-83d5-4539-96fb-abcf974a2180\") " pod="openshift-console/downloads-c65b8dc4b-rjdvs" Jun 03 15:43:31.714388 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.714373 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.717661 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.717638 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 03 15:43:31.718216 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.718178 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 03 15:43:31.718303 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.718216 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 03 15:43:31.724305 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.724285 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4ph84\"" Jun 03 15:43:31.730008 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.729984 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-k2lgj"] Jun 03 15:43:31.731956 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.731938 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 03 15:43:31.732898 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.732881 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.735676 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.735654 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2w7fl\"" Jun 03 15:43:31.735777 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.735697 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 03 15:43:31.735777 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.735662 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 03 15:43:31.740583 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.740562 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58fbcd44d-rw7xq"] Jun 03 15:43:31.741425 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.741407 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 03 15:43:31.741662 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.741640 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmfn7\" (UniqueName: \"kubernetes.io/projected/aafc4657-83d5-4539-96fb-abcf974a2180-kube-api-access-xmfn7\") pod \"downloads-c65b8dc4b-rjdvs\" (UID: \"aafc4657-83d5-4539-96fb-abcf974a2180\") " pod="openshift-console/downloads-c65b8dc4b-rjdvs" Jun 03 15:43:31.743113 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.743094 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 03 15:43:31.743691 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.743669 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k2lgj"] Jun 03 15:43:31.813236 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813183 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d29a2368-495f-4637-8663-32d7687e81ac-crio-socket\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.813236 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813237 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34c38475-cfdd-4e86-ae06-cf991fd4aa53-registry-tls\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.813439 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813259 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d29a2368-495f-4637-8663-32d7687e81ac-data-volume\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.813439 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813292 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d29a2368-495f-4637-8663-32d7687e81ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.813439 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813330 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34c38475-cfdd-4e86-ae06-cf991fd4aa53-trusted-ca\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.813439 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813352 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34c38475-cfdd-4e86-ae06-cf991fd4aa53-registry-certificates\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.813439 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813404 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d29a2368-495f-4637-8663-32d7687e81ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.813439 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbns6\" (UniqueName: \"kubernetes.io/projected/d29a2368-495f-4637-8663-32d7687e81ac-kube-api-access-sbns6\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.813635 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813439 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g77w\" (UniqueName: \"kubernetes.io/projected/34c38475-cfdd-4e86-ae06-cf991fd4aa53-kube-api-access-4g77w\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.813635 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813465 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34c38475-cfdd-4e86-ae06-cf991fd4aa53-installation-pull-secrets\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.813635 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813481 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34c38475-cfdd-4e86-ae06-cf991fd4aa53-bound-sa-token\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.813635 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813533 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34c38475-cfdd-4e86-ae06-cf991fd4aa53-image-registry-private-configuration\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.813635 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.813552 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34c38475-cfdd-4e86-ae06-cf991fd4aa53-ca-trust-extracted\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.913866 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.913793 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d29a2368-495f-4637-8663-32d7687e81ac-crio-socket\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.913866 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.913826 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34c38475-cfdd-4e86-ae06-cf991fd4aa53-registry-tls\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.913866 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.913843 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d29a2368-495f-4637-8663-32d7687e81ac-data-volume\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.913866 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.913860 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d29a2368-495f-4637-8663-32d7687e81ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.914130 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.913902 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34c38475-cfdd-4e86-ae06-cf991fd4aa53-trusted-ca\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.914130 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.913911 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d29a2368-495f-4637-8663-32d7687e81ac-crio-socket\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.914130 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.913949 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34c38475-cfdd-4e86-ae06-cf991fd4aa53-registry-certificates\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.914130 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.914019 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d29a2368-495f-4637-8663-32d7687e81ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.914130 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.914045 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sbns6\" (UniqueName: \"kubernetes.io/projected/d29a2368-495f-4637-8663-32d7687e81ac-kube-api-access-sbns6\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.914130 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.914072 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4g77w\" (UniqueName: \"kubernetes.io/projected/34c38475-cfdd-4e86-ae06-cf991fd4aa53-kube-api-access-4g77w\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.914415 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.914110 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34c38475-cfdd-4e86-ae06-cf991fd4aa53-installation-pull-secrets\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.914415 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.914169 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34c38475-cfdd-4e86-ae06-cf991fd4aa53-bound-sa-token\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.914415 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.914250 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d29a2368-495f-4637-8663-32d7687e81ac-data-volume\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.914415 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.914244 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34c38475-cfdd-4e86-ae06-cf991fd4aa53-image-registry-private-configuration\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.914415 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.914371 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34c38475-cfdd-4e86-ae06-cf991fd4aa53-ca-trust-extracted\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.914651 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.914596 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d29a2368-495f-4637-8663-32d7687e81ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.914972 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.914949 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34c38475-cfdd-4e86-ae06-cf991fd4aa53-ca-trust-extracted\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.915214 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.915173 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34c38475-cfdd-4e86-ae06-cf991fd4aa53-registry-certificates\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.915329 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.915309 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34c38475-cfdd-4e86-ae06-cf991fd4aa53-trusted-ca\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.916466 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.916437 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d29a2368-495f-4637-8663-32d7687e81ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.916702 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.916680 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34c38475-cfdd-4e86-ae06-cf991fd4aa53-registry-tls\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.916812 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.916789 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34c38475-cfdd-4e86-ae06-cf991fd4aa53-image-registry-private-configuration\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.916949 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.916931 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34c38475-cfdd-4e86-ae06-cf991fd4aa53-installation-pull-secrets\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.917750 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.917738 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-rjdvs" Jun 03 15:43:31.928616 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.928438 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g77w\" (UniqueName: \"kubernetes.io/projected/34c38475-cfdd-4e86-ae06-cf991fd4aa53-kube-api-access-4g77w\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:31.928616 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.928496 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbns6\" (UniqueName: \"kubernetes.io/projected/d29a2368-495f-4637-8663-32d7687e81ac-kube-api-access-sbns6\") pod \"insights-runtime-extractor-k2lgj\" (UID: \"d29a2368-495f-4637-8663-32d7687e81ac\") " pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:31.928925 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:31.928881 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34c38475-cfdd-4e86-ae06-cf991fd4aa53-bound-sa-token\") pod \"image-registry-58fbcd44d-rw7xq\" (UID: \"34c38475-cfdd-4e86-ae06-cf991fd4aa53\") " pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:32.022721 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:32.022691 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:32.041388 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:32.041354 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k2lgj" Jun 03 15:43:32.042249 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:32.042054 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-rjdvs"] Jun 03 15:43:32.044927 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:43:32.044895 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaafc4657_83d5_4539_96fb_abcf974a2180.slice/crio-85ebed6f81fc8186439cba343f1452e090b8af35f9f4acb9c0bf36fd7bcdd682 WatchSource:0}: Error finding container 85ebed6f81fc8186439cba343f1452e090b8af35f9f4acb9c0bf36fd7bcdd682: Status 404 returned error can't find the container with id 85ebed6f81fc8186439cba343f1452e090b8af35f9f4acb9c0bf36fd7bcdd682 Jun 03 15:43:32.160638 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:32.160604 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58fbcd44d-rw7xq"] Jun 03 15:43:32.163731 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:43:32.163704 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c38475_cfdd_4e86_ae06_cf991fd4aa53.slice/crio-a6f5da52a784c55266f3d0c339eb9d6c572c501c4e7140350f777d7841c92c24 WatchSource:0}: Error finding container a6f5da52a784c55266f3d0c339eb9d6c572c501c4e7140350f777d7841c92c24: Status 404 returned error can't find the container with id a6f5da52a784c55266f3d0c339eb9d6c572c501c4e7140350f777d7841c92c24 Jun 03 15:43:32.181585 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:32.181566 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k2lgj"] Jun 03 15:43:32.183774 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:43:32.183748 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29a2368_495f_4637_8663_32d7687e81ac.slice/crio-2f08d8418f230a0b74424bf2586514d42ffb1fa038d83087b626c6adaa34a30b WatchSource:0}: Error finding container 2f08d8418f230a0b74424bf2586514d42ffb1fa038d83087b626c6adaa34a30b: Status 404 returned error can't find the container with id 2f08d8418f230a0b74424bf2586514d42ffb1fa038d83087b626c6adaa34a30b Jun 03 15:43:32.581748 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:32.581710 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k2lgj" event={"ID":"d29a2368-495f-4637-8663-32d7687e81ac","Type":"ContainerStarted","Data":"d6837232df4d3e7fcf62ed3bd80e4d1a2223e002b6be4c4e600e5bf583031321"} Jun 03 15:43:32.581748 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:32.581752 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k2lgj" event={"ID":"d29a2368-495f-4637-8663-32d7687e81ac","Type":"ContainerStarted","Data":"2f08d8418f230a0b74424bf2586514d42ffb1fa038d83087b626c6adaa34a30b"} Jun 03 15:43:32.583019 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:32.582992 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" event={"ID":"34c38475-cfdd-4e86-ae06-cf991fd4aa53","Type":"ContainerStarted","Data":"26e700ead4393f29cfaf8060e4a67a61421ce1d1268b88d811a3a7b8dd7912b0"} Jun 03 15:43:32.583150 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:32.583024 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" event={"ID":"34c38475-cfdd-4e86-ae06-cf991fd4aa53","Type":"ContainerStarted","Data":"a6f5da52a784c55266f3d0c339eb9d6c572c501c4e7140350f777d7841c92c24"} Jun 03 15:43:32.583150 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:32.583081 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:43:32.584085 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:32.584061 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-rjdvs" event={"ID":"aafc4657-83d5-4539-96fb-abcf974a2180","Type":"ContainerStarted","Data":"85ebed6f81fc8186439cba343f1452e090b8af35f9f4acb9c0bf36fd7bcdd682"} Jun 03 15:43:32.612501 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:32.612458 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" podStartSLOduration=1.612445087 podStartE2EDuration="1.612445087s" podCreationTimestamp="2026-06-03 15:43:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 15:43:32.610631619 +0000 UTC m=+159.995698838" watchObservedRunningTime="2026-06-03 15:43:32.612445087 +0000 UTC m=+159.997512303" Jun 03 15:43:33.588003 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:33.587943 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k2lgj" event={"ID":"d29a2368-495f-4637-8663-32d7687e81ac","Type":"ContainerStarted","Data":"dbe54c7d5b91778342a3527fe67bb95c3bfbdfc5b9e7ab4ce7e4527344559ddf"} Jun 03 15:43:35.445687 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:35.445647 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:43:35.448292 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:35.448267 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cb3b438-55f9-4194-ba2a-e91afec5a1e4-metrics-tls\") pod \"dns-default-n4lvz\" (UID: \"6cb3b438-55f9-4194-ba2a-e91afec5a1e4\") " pod="openshift-dns/dns-default-n4lvz" Jun 03 15:43:35.482719 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:35.482687 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2z9xs\"" Jun 03 15:43:35.489804 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:35.489778 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n4lvz" Jun 03 15:43:35.546888 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:35.546843 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:43:35.551039 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:35.550809 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca43b94-b17e-4492-a995-30971a21f97b-cert\") pod \"ingress-canary-96wm8\" (UID: \"5ca43b94-b17e-4492-a995-30971a21f97b\") " pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:43:35.596375 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:35.596332 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k2lgj" event={"ID":"d29a2368-495f-4637-8663-32d7687e81ac","Type":"ContainerStarted","Data":"23f469ac0d5244e2f9660a508e37a64114c201ea370fd8a2cefa50352da5beb0"} Jun 03 15:43:35.628134 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:35.628074 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-k2lgj" podStartSLOduration=2.2748219020000002 podStartE2EDuration="4.628055704s" podCreationTimestamp="2026-06-03 15:43:31 +0000 UTC" firstStartedPulling="2026-06-03 15:43:32.233671129 +0000 UTC m=+159.618738326" lastFinishedPulling="2026-06-03 15:43:34.586904922 +0000 UTC m=+161.971972128" observedRunningTime="2026-06-03 15:43:35.627380647 +0000 UTC m=+163.012447869" watchObservedRunningTime="2026-06-03 15:43:35.628055704 +0000 UTC m=+163.013122924" Jun 03 15:43:35.640505 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:35.640476 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n4lvz"] Jun 03 15:43:35.644183 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:43:35.644152 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb3b438_55f9_4194_ba2a_e91afec5a1e4.slice/crio-80bf855657c7906fd9beb554d44e0a49ef3b1bebb86a55604431beb901fd5eec WatchSource:0}: Error finding container 80bf855657c7906fd9beb554d44e0a49ef3b1bebb86a55604431beb901fd5eec: Status 404 returned error can't find the container with id 80bf855657c7906fd9beb554d44e0a49ef3b1bebb86a55604431beb901fd5eec Jun 03 15:43:36.600074 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:36.600032 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n4lvz" event={"ID":"6cb3b438-55f9-4194-ba2a-e91afec5a1e4","Type":"ContainerStarted","Data":"80bf855657c7906fd9beb554d44e0a49ef3b1bebb86a55604431beb901fd5eec"} Jun 03 15:43:37.604079 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:37.604041 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n4lvz" event={"ID":"6cb3b438-55f9-4194-ba2a-e91afec5a1e4","Type":"ContainerStarted","Data":"e115b2ec4c38afbab103e87d2cb34d85ea3604788cde389669f7ae7592037a97"} Jun 03 15:43:37.604079 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:37.604086 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n4lvz" event={"ID":"6cb3b438-55f9-4194-ba2a-e91afec5a1e4","Type":"ContainerStarted","Data":"a48ebc9b879ea860f587513e1729a73937cc71fc6a7fd26e1d1b312e46bf5252"} Jun 03 15:43:37.604580 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:37.604188 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n4lvz" Jun 03 15:43:37.625217 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:37.625147 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-n4lvz" podStartSLOduration=129.254782712 podStartE2EDuration="2m10.625061935s" podCreationTimestamp="2026-06-03 15:41:27 +0000 UTC" firstStartedPulling="2026-06-03 15:43:35.646497616 +0000 UTC m=+163.031564816" lastFinishedPulling="2026-06-03 15:43:37.016776837 +0000 UTC m=+164.401844039" observedRunningTime="2026-06-03 15:43:37.623973926 +0000 UTC m=+165.009041143" watchObservedRunningTime="2026-06-03 15:43:37.625061935 +0000 UTC m=+165.010129153" Jun 03 15:43:39.324258 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:39.324185 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" podUID="7f967aa1-5577-4dc3-b7dc-7229afe3f3b1" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.7:8000/readyz\": dial tcp 10.134.0.7:8000: connect: connection refused" Jun 03 15:43:39.610210 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:39.610116 2578 generic.go:358] "Generic (PLEG): container finished" podID="7f967aa1-5577-4dc3-b7dc-7229afe3f3b1" containerID="21c51700f7a8d0873c8c29b7297ed473f1e935b0d305ac39aef71e75774f11b0" exitCode=1 Jun 03 15:43:39.610210 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:39.610157 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" event={"ID":"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1","Type":"ContainerDied","Data":"21c51700f7a8d0873c8c29b7297ed473f1e935b0d305ac39aef71e75774f11b0"} Jun 03 15:43:39.610464 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:39.610451 2578 scope.go:117] "RemoveContainer" containerID="21c51700f7a8d0873c8c29b7297ed473f1e935b0d305ac39aef71e75774f11b0" Jun 03 15:43:40.614958 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:40.614920 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" event={"ID":"7f967aa1-5577-4dc3-b7dc-7229afe3f3b1","Type":"ContainerStarted","Data":"2bf8bd30f70131e9c326ac6a16080f738b37131e687c7ba3b9783a59c16cb52c"} Jun 03 15:43:40.615356 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:40.615212 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:43:40.615757 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:40.615741 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-575c4bdffd-62z5x" Jun 03 15:43:44.791070 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.791034 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5txw4"] Jun 03 15:43:44.793617 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.793591 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:44.795827 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.795803 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 03 15:43:44.797192 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.797168 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 03 15:43:44.797312 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.797172 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 03 15:43:44.797312 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.797181 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 03 15:43:44.797634 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.797615 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 03 15:43:44.798048 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.798028 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-blgxq\"" Jun 03 15:43:44.802448 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.798553 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 03 15:43:44.919086 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.919044 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-tls\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:44.919301 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.919115 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-sys\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:44.919301 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.919141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-wtmp\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:44.919301 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.919167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-accelerators-collector-config\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:44.919301 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.919222 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgsjl\" (UniqueName: \"kubernetes.io/projected/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-kube-api-access-hgsjl\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:44.919301 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.919241 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-root\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:44.919301 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.919280 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:44.919553 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.919375 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-metrics-client-ca\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:44.919553 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:44.919419 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-textfile\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.020674 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.020634 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-tls\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.020869 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.020689 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-sys\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.020869 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.020723 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-wtmp\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.020869 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.020756 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-accelerators-collector-config\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.020869 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.020812 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hgsjl\" (UniqueName: \"kubernetes.io/projected/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-kube-api-access-hgsjl\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.020869 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.020840 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-root\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.020869 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.020839 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-sys\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.021175 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.020884 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.021175 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.020906 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-root\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.021175 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.020932 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-metrics-client-ca\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.021175 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.020935 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-wtmp\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.021175 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.020959 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-textfile\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.021681 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.021618 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-accelerators-collector-config\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.021838 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.021821 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-textfile\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.022178 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.022157 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-metrics-client-ca\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.023514 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.023489 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-tls\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.023614 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.023527 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.036881 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.036850 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgsjl\" (UniqueName: \"kubernetes.io/projected/6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3-kube-api-access-hgsjl\") pod \"node-exporter-5txw4\" (UID: \"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3\") " pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.104956 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.104878 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5txw4" Jun 03 15:43:45.107803 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.107776 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:43:45.107932 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.107776 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:43:45.113513 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.113493 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-f2mct\"" Jun 03 15:43:45.118693 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:45.118676 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-96wm8" Jun 03 15:43:47.609636 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:47.609604 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n4lvz" Jun 03 15:43:49.888060 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:43:49.888020 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6082d5ec_f4d3_4d2e_b4cd_aa32c27f1da3.slice/crio-f5d405cdccdea5106cfe1ee961c63118e371c471416a0d5ce33cbe480ed1cace WatchSource:0}: Error finding container f5d405cdccdea5106cfe1ee961c63118e371c471416a0d5ce33cbe480ed1cace: Status 404 returned error can't find the container with id f5d405cdccdea5106cfe1ee961c63118e371c471416a0d5ce33cbe480ed1cace Jun 03 15:43:50.028537 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:50.028503 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-96wm8"] Jun 03 15:43:50.031836 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:43:50.031807 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ca43b94_b17e_4492_a995_30971a21f97b.slice/crio-f762d026398bddc2797fc0735c51798a0dacc930054c9d63c2e54abbc8f9d1bc WatchSource:0}: Error finding container f762d026398bddc2797fc0735c51798a0dacc930054c9d63c2e54abbc8f9d1bc: Status 404 returned error can't find the container with id f762d026398bddc2797fc0735c51798a0dacc930054c9d63c2e54abbc8f9d1bc Jun 03 15:43:50.641588 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:50.641533 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5txw4" event={"ID":"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3","Type":"ContainerStarted","Data":"f5d405cdccdea5106cfe1ee961c63118e371c471416a0d5ce33cbe480ed1cace"} Jun 03 15:43:50.643406 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:50.643359 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-rjdvs" event={"ID":"aafc4657-83d5-4539-96fb-abcf974a2180","Type":"ContainerStarted","Data":"60b7a978dc8aea316174fea53905839295e9ca4ef7e4b38e18977235f9680c8d"} Jun 03 15:43:50.644815 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:50.644768 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-96wm8" event={"ID":"5ca43b94-b17e-4492-a995-30971a21f97b","Type":"ContainerStarted","Data":"f762d026398bddc2797fc0735c51798a0dacc930054c9d63c2e54abbc8f9d1bc"} Jun 03 15:43:50.667705 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:50.667646 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c65b8dc4b-rjdvs" podStartSLOduration=1.276922632 podStartE2EDuration="19.667632187s" podCreationTimestamp="2026-06-03 15:43:31 +0000 UTC" firstStartedPulling="2026-06-03 15:43:32.046752848 +0000 UTC m=+159.431820046" lastFinishedPulling="2026-06-03 15:43:50.437462389 +0000 UTC m=+177.822529601" observedRunningTime="2026-06-03 15:43:50.667229455 +0000 UTC m=+178.052296675" watchObservedRunningTime="2026-06-03 15:43:50.667632187 +0000 UTC m=+178.052699406" Jun 03 15:43:51.649970 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:51.649927 2578 generic.go:358] "Generic (PLEG): container finished" podID="6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3" containerID="7b9e49706a69c1e4aff5744c26ac06b07e8aab314603526e61e70d0200b376b8" exitCode=0 Jun 03 15:43:51.650522 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:51.650171 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5txw4" event={"ID":"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3","Type":"ContainerDied","Data":"7b9e49706a69c1e4aff5744c26ac06b07e8aab314603526e61e70d0200b376b8"} Jun 03 15:43:51.650522 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:51.650283 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c65b8dc4b-rjdvs" Jun 03 15:43:51.660762 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:51.660701 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c65b8dc4b-rjdvs" Jun 03 15:43:52.654215 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:52.654167 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-96wm8" event={"ID":"5ca43b94-b17e-4492-a995-30971a21f97b","Type":"ContainerStarted","Data":"58f5c7359bf80f4ecce52dc75e706cb0bf825698bdc1dd5fdf7e2ab26e03c53d"} Jun 03 15:43:52.656602 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:52.656572 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5txw4" event={"ID":"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3","Type":"ContainerStarted","Data":"a666e0fe8f0a08c299389163c81454b60f955e43727e7f9ba2e6b239ab889bf5"} Jun 03 15:43:52.656742 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:52.656609 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5txw4" event={"ID":"6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3","Type":"ContainerStarted","Data":"262b399b54dc78b1038138a9e5ad78b449fb522aff02657f4a78ff099690b8a8"} Jun 03 15:43:52.678844 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:52.678778 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-96wm8" podStartSLOduration=143.771693298 podStartE2EDuration="2m25.678757419s" podCreationTimestamp="2026-06-03 15:41:27 +0000 UTC" firstStartedPulling="2026-06-03 15:43:50.033707928 +0000 UTC m=+177.418775125" lastFinishedPulling="2026-06-03 15:43:51.940772044 +0000 UTC m=+179.325839246" observedRunningTime="2026-06-03 15:43:52.677494836 +0000 UTC m=+180.062562067" watchObservedRunningTime="2026-06-03 15:43:52.678757419 +0000 UTC m=+180.063824639" Jun 03 15:43:52.717025 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:52.716970 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5txw4" podStartSLOduration=7.895352543 podStartE2EDuration="8.716955901s" podCreationTimestamp="2026-06-03 15:43:44 +0000 UTC" firstStartedPulling="2026-06-03 15:43:49.889868297 +0000 UTC m=+177.274935497" lastFinishedPulling="2026-06-03 15:43:50.711471648 +0000 UTC m=+178.096538855" observedRunningTime="2026-06-03 15:43:52.71583284 +0000 UTC m=+180.100900080" watchObservedRunningTime="2026-06-03 15:43:52.716955901 +0000 UTC m=+180.102023144" Jun 03 15:43:53.592879 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:43:53.592844 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-58fbcd44d-rw7xq" Jun 03 15:44:12.988785 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:44:12.988742 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" podUID="1f6a6e61-a7cf-4552-8859-3896315b5774" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 03 15:44:22.989236 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:44:22.989171 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" podUID="1f6a6e61-a7cf-4552-8859-3896315b5774" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 03 15:44:32.988270 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:44:32.988234 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" podUID="1f6a6e61-a7cf-4552-8859-3896315b5774" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 03 15:44:32.988622 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:44:32.988296 2578 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" Jun 03 15:44:32.989247 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:44:32.988738 2578 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"3605b0a0ed742ecbc517746f0d3de74d2048fa3098efc17f838fdae3597c868f"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 03 15:44:32.989247 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:44:32.988808 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" podUID="1f6a6e61-a7cf-4552-8859-3896315b5774" containerName="service-proxy" containerID="cri-o://3605b0a0ed742ecbc517746f0d3de74d2048fa3098efc17f838fdae3597c868f" gracePeriod=30 Jun 03 15:44:33.761823 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:44:33.761787 2578 generic.go:358] "Generic (PLEG): container finished" podID="1f6a6e61-a7cf-4552-8859-3896315b5774" containerID="3605b0a0ed742ecbc517746f0d3de74d2048fa3098efc17f838fdae3597c868f" exitCode=2 Jun 03 15:44:33.761991 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:44:33.761862 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" event={"ID":"1f6a6e61-a7cf-4552-8859-3896315b5774","Type":"ContainerDied","Data":"3605b0a0ed742ecbc517746f0d3de74d2048fa3098efc17f838fdae3597c868f"} Jun 03 15:44:33.761991 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:44:33.761895 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69c9c66c-lp4pn" event={"ID":"1f6a6e61-a7cf-4552-8859-3896315b5774","Type":"ContainerStarted","Data":"5c14320fe893fcc7a83fd89c1424cc6f467d9a71ccbb2f1d862005301acf902d"} Jun 03 15:45:04.835803 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:04.835705 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:45:04.837994 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:04.837970 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9d8c3c0-82e0-48b5-909a-a7887d92baa9-metrics-certs\") pod \"network-metrics-daemon-t7kx5\" (UID: \"a9d8c3c0-82e0-48b5-909a-a7887d92baa9\") " pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:45:04.911591 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:04.911559 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q8ntl\"" Jun 03 15:45:04.918870 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:04.918842 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t7kx5" Jun 03 15:45:05.047921 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:05.047885 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t7kx5"] Jun 03 15:45:05.051089 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:45:05.051063 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9d8c3c0_82e0_48b5_909a_a7887d92baa9.slice/crio-580c1d9efb1f9482bf144e294e6d125fbf20254e8874e7bde68cbad3d81a82cb WatchSource:0}: Error finding container 580c1d9efb1f9482bf144e294e6d125fbf20254e8874e7bde68cbad3d81a82cb: Status 404 returned error can't find the container with id 580c1d9efb1f9482bf144e294e6d125fbf20254e8874e7bde68cbad3d81a82cb Jun 03 15:45:05.848974 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:05.848936 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t7kx5" event={"ID":"a9d8c3c0-82e0-48b5-909a-a7887d92baa9","Type":"ContainerStarted","Data":"580c1d9efb1f9482bf144e294e6d125fbf20254e8874e7bde68cbad3d81a82cb"} Jun 03 15:45:06.852733 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:06.852693 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t7kx5" event={"ID":"a9d8c3c0-82e0-48b5-909a-a7887d92baa9","Type":"ContainerStarted","Data":"90438dccfa3c1fc6a36ddf3d6833348312e23f042d8ff81ef1cc8c5380fcf5b5"} Jun 03 15:45:06.852733 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:06.852729 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t7kx5" event={"ID":"a9d8c3c0-82e0-48b5-909a-a7887d92baa9","Type":"ContainerStarted","Data":"6d72125e39aba04140dd910fc03662c0c1cf11fc7c1d701e66271a6eb86b605b"} Jun 03 15:45:06.872654 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:06.872591 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t7kx5" podStartSLOduration=252.954743084 podStartE2EDuration="4m13.872572659s" podCreationTimestamp="2026-06-03 15:40:53 +0000 UTC" firstStartedPulling="2026-06-03 15:45:05.052855902 +0000 UTC m=+252.437923102" lastFinishedPulling="2026-06-03 15:45:05.970685472 +0000 UTC m=+253.355752677" observedRunningTime="2026-06-03 15:45:06.871402935 +0000 UTC m=+254.256470155" watchObservedRunningTime="2026-06-03 15:45:06.872572659 +0000 UTC m=+254.257639882" Jun 03 15:45:39.188377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.188339 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-f27dk"] Jun 03 15:45:39.191701 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.191678 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f27dk" Jun 03 15:45:39.194771 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.194748 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 03 15:45:39.203872 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.203845 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f27dk"] Jun 03 15:45:39.275982 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.275942 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/576c5214-8806-4fd0-896c-0b055986dab4-original-pull-secret\") pod \"global-pull-secret-syncer-f27dk\" (UID: \"576c5214-8806-4fd0-896c-0b055986dab4\") " pod="kube-system/global-pull-secret-syncer-f27dk" Jun 03 15:45:39.275982 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.275989 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/576c5214-8806-4fd0-896c-0b055986dab4-dbus\") pod \"global-pull-secret-syncer-f27dk\" (UID: \"576c5214-8806-4fd0-896c-0b055986dab4\") " pod="kube-system/global-pull-secret-syncer-f27dk" Jun 03 15:45:39.276235 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.276082 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/576c5214-8806-4fd0-896c-0b055986dab4-kubelet-config\") pod \"global-pull-secret-syncer-f27dk\" (UID: \"576c5214-8806-4fd0-896c-0b055986dab4\") " pod="kube-system/global-pull-secret-syncer-f27dk" Jun 03 15:45:39.376809 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.376760 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/576c5214-8806-4fd0-896c-0b055986dab4-original-pull-secret\") pod \"global-pull-secret-syncer-f27dk\" (UID: \"576c5214-8806-4fd0-896c-0b055986dab4\") " pod="kube-system/global-pull-secret-syncer-f27dk" Jun 03 15:45:39.376809 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.376810 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/576c5214-8806-4fd0-896c-0b055986dab4-dbus\") pod \"global-pull-secret-syncer-f27dk\" (UID: \"576c5214-8806-4fd0-896c-0b055986dab4\") " pod="kube-system/global-pull-secret-syncer-f27dk" Jun 03 15:45:39.377078 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.376846 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/576c5214-8806-4fd0-896c-0b055986dab4-kubelet-config\") pod \"global-pull-secret-syncer-f27dk\" (UID: \"576c5214-8806-4fd0-896c-0b055986dab4\") " pod="kube-system/global-pull-secret-syncer-f27dk" Jun 03 15:45:39.377078 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.376942 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/576c5214-8806-4fd0-896c-0b055986dab4-kubelet-config\") pod \"global-pull-secret-syncer-f27dk\" (UID: \"576c5214-8806-4fd0-896c-0b055986dab4\") " pod="kube-system/global-pull-secret-syncer-f27dk" Jun 03 15:45:39.377078 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.377025 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/576c5214-8806-4fd0-896c-0b055986dab4-dbus\") pod \"global-pull-secret-syncer-f27dk\" (UID: \"576c5214-8806-4fd0-896c-0b055986dab4\") " pod="kube-system/global-pull-secret-syncer-f27dk" Jun 03 15:45:39.379143 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.379122 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/576c5214-8806-4fd0-896c-0b055986dab4-original-pull-secret\") pod \"global-pull-secret-syncer-f27dk\" (UID: \"576c5214-8806-4fd0-896c-0b055986dab4\") " pod="kube-system/global-pull-secret-syncer-f27dk" Jun 03 15:45:39.501183 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.501154 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f27dk" Jun 03 15:45:39.631473 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.631445 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f27dk"] Jun 03 15:45:39.633734 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:45:39.633708 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod576c5214_8806_4fd0_896c_0b055986dab4.slice/crio-a989aa4301afc4a0bf16d09d7591d82c4a228a9f7177826cab9481a28880001a WatchSource:0}: Error finding container a989aa4301afc4a0bf16d09d7591d82c4a228a9f7177826cab9481a28880001a: Status 404 returned error can't find the container with id a989aa4301afc4a0bf16d09d7591d82c4a228a9f7177826cab9481a28880001a Jun 03 15:45:39.938631 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:39.938544 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f27dk" event={"ID":"576c5214-8806-4fd0-896c-0b055986dab4","Type":"ContainerStarted","Data":"a989aa4301afc4a0bf16d09d7591d82c4a228a9f7177826cab9481a28880001a"} Jun 03 15:45:44.953505 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:44.953471 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f27dk" event={"ID":"576c5214-8806-4fd0-896c-0b055986dab4","Type":"ContainerStarted","Data":"3a17eefacd978ace223780b776edb91b30c1725d57ce4640dd65a1e4b52c8a5b"} Jun 03 15:45:44.970317 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:44.970271 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f27dk" podStartSLOduration=1.403922676 podStartE2EDuration="5.970258127s" podCreationTimestamp="2026-06-03 15:45:39 +0000 UTC" firstStartedPulling="2026-06-03 15:45:39.635393041 +0000 UTC m=+287.020460238" lastFinishedPulling="2026-06-03 15:45:44.201728488 +0000 UTC m=+291.586795689" observedRunningTime="2026-06-03 15:45:44.968779645 +0000 UTC m=+292.353846864" watchObservedRunningTime="2026-06-03 15:45:44.970258127 +0000 UTC m=+292.355325345" Jun 03 15:45:52.984373 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:52.984346 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 15:45:52.984806 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:52.984556 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 15:45:52.995951 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:45:52.995924 2578 kubelet.go:1628] "Image garbage collection succeeded" Jun 03 15:47:03.503340 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.503298 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh"] Jun 03 15:47:03.506431 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.506407 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.509989 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.509965 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 03 15:47:03.510255 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.510241 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fbrvd\"" Jun 03 15:47:03.510603 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.510586 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 03 15:47:03.510875 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.510862 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 03 15:47:03.512689 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.512668 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 03 15:47:03.520741 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.520716 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh"] Jun 03 15:47:03.568994 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.568952 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqzws\" (UniqueName: \"kubernetes.io/projected/b23bd02b-221f-4d20-8887-430a22bf1f39-kube-api-access-vqzws\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.568994 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.568999 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b23bd02b-221f-4d20-8887-430a22bf1f39-manifests\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.569263 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.569065 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b23bd02b-221f-4d20-8887-430a22bf1f39-webhook-cert\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.569263 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.569139 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b23bd02b-221f-4d20-8887-430a22bf1f39-tmp\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.569263 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.569176 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b23bd02b-221f-4d20-8887-430a22bf1f39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.669954 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.669904 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b23bd02b-221f-4d20-8887-430a22bf1f39-webhook-cert\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.669954 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.669955 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b23bd02b-221f-4d20-8887-430a22bf1f39-tmp\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.670226 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.669982 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b23bd02b-221f-4d20-8887-430a22bf1f39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.670226 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.670023 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vqzws\" (UniqueName: \"kubernetes.io/projected/b23bd02b-221f-4d20-8887-430a22bf1f39-kube-api-access-vqzws\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.670226 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.670052 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b23bd02b-221f-4d20-8887-430a22bf1f39-manifests\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.672684 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.670896 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b23bd02b-221f-4d20-8887-430a22bf1f39-manifests\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.675164 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.675136 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b23bd02b-221f-4d20-8887-430a22bf1f39-tmp\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.675479 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.675447 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b23bd02b-221f-4d20-8887-430a22bf1f39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.675577 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.675535 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b23bd02b-221f-4d20-8887-430a22bf1f39-webhook-cert\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.681540 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.681513 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqzws\" (UniqueName: \"kubernetes.io/projected/b23bd02b-221f-4d20-8887-430a22bf1f39-kube-api-access-vqzws\") pod \"opendatahub-operator-controller-manager-548f4487f9-dnxvh\" (UID: \"b23bd02b-221f-4d20-8887-430a22bf1f39\") " pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.817254 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.817145 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:03.966191 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.966155 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh"] Jun 03 15:47:03.968795 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:47:03.968769 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23bd02b_221f_4d20_8887_430a22bf1f39.slice/crio-d0783721fc993e020d926167ea361303c689e23c69d13dd6d2a5fe12007615e6 WatchSource:0}: Error finding container d0783721fc993e020d926167ea361303c689e23c69d13dd6d2a5fe12007615e6: Status 404 returned error can't find the container with id d0783721fc993e020d926167ea361303c689e23c69d13dd6d2a5fe12007615e6 Jun 03 15:47:03.970407 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:03.970392 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 15:47:04.153081 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:04.152991 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" event={"ID":"b23bd02b-221f-4d20-8887-430a22bf1f39","Type":"ContainerStarted","Data":"d0783721fc993e020d926167ea361303c689e23c69d13dd6d2a5fe12007615e6"} Jun 03 15:47:07.161831 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:07.161746 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" event={"ID":"b23bd02b-221f-4d20-8887-430a22bf1f39","Type":"ContainerStarted","Data":"1afbe63cc1cea4a0797e3419cd87b19d22561a1de0909094c48c204dd89ce2b4"} Jun 03 15:47:08.165421 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:08.165325 2578 generic.go:358] "Generic (PLEG): container finished" podID="b23bd02b-221f-4d20-8887-430a22bf1f39" containerID="1afbe63cc1cea4a0797e3419cd87b19d22561a1de0909094c48c204dd89ce2b4" exitCode=0 Jun 03 15:47:08.165421 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:08.165409 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" event={"ID":"b23bd02b-221f-4d20-8887-430a22bf1f39","Type":"ContainerDied","Data":"1afbe63cc1cea4a0797e3419cd87b19d22561a1de0909094c48c204dd89ce2b4"} Jun 03 15:47:09.170664 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:09.170623 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" event={"ID":"b23bd02b-221f-4d20-8887-430a22bf1f39","Type":"ContainerStarted","Data":"e6611a8d49d94ebf056c68b423ce7bd6c65f1aa9ef28e46b9831d2eb21a1fac4"} Jun 03 15:47:09.170664 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:09.170674 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:09.200452 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:09.200390 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" podStartSLOduration=1.6755214029999999 podStartE2EDuration="6.200371196s" podCreationTimestamp="2026-06-03 15:47:03 +0000 UTC" firstStartedPulling="2026-06-03 15:47:03.970513763 +0000 UTC m=+371.355580960" lastFinishedPulling="2026-06-03 15:47:08.495363552 +0000 UTC m=+375.880430753" observedRunningTime="2026-06-03 15:47:09.19836784 +0000 UTC m=+376.583435061" watchObservedRunningTime="2026-06-03 15:47:09.200371196 +0000 UTC m=+376.585438418" Jun 03 15:47:20.175935 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:20.175902 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-548f4487f9-dnxvh" Jun 03 15:47:23.334923 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:23.334882 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-76588b95-jwgq2"] Jun 03 15:47:23.347363 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:23.347331 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-jwgq2" Jun 03 15:47:23.349926 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:23.349896 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 03 15:47:23.350049 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:23.349991 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ztkb6\"" Jun 03 15:47:23.354685 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:23.353505 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-jwgq2"] Jun 03 15:47:23.421163 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:23.421122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57fd6338-2ffe-40bc-a9f4-5399b6bb441d-cert\") pod \"odh-model-controller-76588b95-jwgq2\" (UID: \"57fd6338-2ffe-40bc-a9f4-5399b6bb441d\") " pod="opendatahub/odh-model-controller-76588b95-jwgq2" Jun 03 15:47:23.421357 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:23.421174 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77tns\" (UniqueName: \"kubernetes.io/projected/57fd6338-2ffe-40bc-a9f4-5399b6bb441d-kube-api-access-77tns\") pod \"odh-model-controller-76588b95-jwgq2\" (UID: \"57fd6338-2ffe-40bc-a9f4-5399b6bb441d\") " pod="opendatahub/odh-model-controller-76588b95-jwgq2" Jun 03 15:47:23.522658 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:23.522616 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57fd6338-2ffe-40bc-a9f4-5399b6bb441d-cert\") pod \"odh-model-controller-76588b95-jwgq2\" (UID: \"57fd6338-2ffe-40bc-a9f4-5399b6bb441d\") " pod="opendatahub/odh-model-controller-76588b95-jwgq2" Jun 03 15:47:23.522843 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:23.522675 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-77tns\" (UniqueName: \"kubernetes.io/projected/57fd6338-2ffe-40bc-a9f4-5399b6bb441d-kube-api-access-77tns\") pod \"odh-model-controller-76588b95-jwgq2\" (UID: \"57fd6338-2ffe-40bc-a9f4-5399b6bb441d\") " pod="opendatahub/odh-model-controller-76588b95-jwgq2" Jun 03 15:47:23.522843 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:47:23.522766 2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 03 15:47:23.522843 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:47:23.522832 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57fd6338-2ffe-40bc-a9f4-5399b6bb441d-cert podName:57fd6338-2ffe-40bc-a9f4-5399b6bb441d nodeName:}" failed. No retries permitted until 2026-06-03 15:47:24.022815163 +0000 UTC m=+391.407882360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/57fd6338-2ffe-40bc-a9f4-5399b6bb441d-cert") pod "odh-model-controller-76588b95-jwgq2" (UID: "57fd6338-2ffe-40bc-a9f4-5399b6bb441d") : secret "odh-model-controller-webhook-cert" not found Jun 03 15:47:23.537304 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:23.537262 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77tns\" (UniqueName: \"kubernetes.io/projected/57fd6338-2ffe-40bc-a9f4-5399b6bb441d-kube-api-access-77tns\") pod \"odh-model-controller-76588b95-jwgq2\" (UID: \"57fd6338-2ffe-40bc-a9f4-5399b6bb441d\") " pod="opendatahub/odh-model-controller-76588b95-jwgq2" Jun 03 15:47:24.028585 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.028550 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57fd6338-2ffe-40bc-a9f4-5399b6bb441d-cert\") pod \"odh-model-controller-76588b95-jwgq2\" (UID: \"57fd6338-2ffe-40bc-a9f4-5399b6bb441d\") " pod="opendatahub/odh-model-controller-76588b95-jwgq2" Jun 03 15:47:24.030738 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.030713 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57fd6338-2ffe-40bc-a9f4-5399b6bb441d-cert\") pod \"odh-model-controller-76588b95-jwgq2\" (UID: \"57fd6338-2ffe-40bc-a9f4-5399b6bb441d\") " pod="opendatahub/odh-model-controller-76588b95-jwgq2" Jun 03 15:47:24.178147 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.178104 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-6577596cd5-bmvnr"] Jun 03 15:47:24.180573 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.180552 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-bmvnr" Jun 03 15:47:24.184371 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.184346 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 03 15:47:24.195904 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.195878 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-bmvnr"] Jun 03 15:47:24.258400 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.258353 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-jwgq2" Jun 03 15:47:24.331008 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.330978 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbcdca8b-8b39-46f0-ac72-64ac11529f65-tls-certs\") pod \"model-serving-api-6577596cd5-bmvnr\" (UID: \"fbcdca8b-8b39-46f0-ac72-64ac11529f65\") " pod="opendatahub/model-serving-api-6577596cd5-bmvnr" Jun 03 15:47:24.331145 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.331021 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpv2l\" (UniqueName: \"kubernetes.io/projected/fbcdca8b-8b39-46f0-ac72-64ac11529f65-kube-api-access-vpv2l\") pod \"model-serving-api-6577596cd5-bmvnr\" (UID: \"fbcdca8b-8b39-46f0-ac72-64ac11529f65\") " pod="opendatahub/model-serving-api-6577596cd5-bmvnr" Jun 03 15:47:24.418092 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.418059 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-jwgq2"] Jun 03 15:47:24.421170 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:47:24.421147 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57fd6338_2ffe_40bc_a9f4_5399b6bb441d.slice/crio-1c4dd5cf4dabcc9ce4c1d09122d7f3f4c7fc3f3460a289449d31524160ebec96 WatchSource:0}: Error finding container 1c4dd5cf4dabcc9ce4c1d09122d7f3f4c7fc3f3460a289449d31524160ebec96: Status 404 returned error can't find the container with id 1c4dd5cf4dabcc9ce4c1d09122d7f3f4c7fc3f3460a289449d31524160ebec96 Jun 03 15:47:24.432002 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.431978 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbcdca8b-8b39-46f0-ac72-64ac11529f65-tls-certs\") pod \"model-serving-api-6577596cd5-bmvnr\" (UID: \"fbcdca8b-8b39-46f0-ac72-64ac11529f65\") " pod="opendatahub/model-serving-api-6577596cd5-bmvnr" Jun 03 15:47:24.432099 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.432016 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vpv2l\" (UniqueName: \"kubernetes.io/projected/fbcdca8b-8b39-46f0-ac72-64ac11529f65-kube-api-access-vpv2l\") pod \"model-serving-api-6577596cd5-bmvnr\" (UID: \"fbcdca8b-8b39-46f0-ac72-64ac11529f65\") " pod="opendatahub/model-serving-api-6577596cd5-bmvnr" Jun 03 15:47:24.432148 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:47:24.432126 2578 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 03 15:47:24.432255 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:47:24.432193 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbcdca8b-8b39-46f0-ac72-64ac11529f65-tls-certs podName:fbcdca8b-8b39-46f0-ac72-64ac11529f65 nodeName:}" failed. No retries permitted until 2026-06-03 15:47:24.932177187 +0000 UTC m=+392.317244388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/fbcdca8b-8b39-46f0-ac72-64ac11529f65-tls-certs") pod "model-serving-api-6577596cd5-bmvnr" (UID: "fbcdca8b-8b39-46f0-ac72-64ac11529f65") : secret "model-serving-api-tls" not found Jun 03 15:47:24.444555 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.444527 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpv2l\" (UniqueName: \"kubernetes.io/projected/fbcdca8b-8b39-46f0-ac72-64ac11529f65-kube-api-access-vpv2l\") pod \"model-serving-api-6577596cd5-bmvnr\" (UID: \"fbcdca8b-8b39-46f0-ac72-64ac11529f65\") " pod="opendatahub/model-serving-api-6577596cd5-bmvnr" Jun 03 15:47:24.937157 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.937107 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbcdca8b-8b39-46f0-ac72-64ac11529f65-tls-certs\") pod \"model-serving-api-6577596cd5-bmvnr\" (UID: \"fbcdca8b-8b39-46f0-ac72-64ac11529f65\") " pod="opendatahub/model-serving-api-6577596cd5-bmvnr" Jun 03 15:47:24.940428 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:24.940398 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbcdca8b-8b39-46f0-ac72-64ac11529f65-tls-certs\") pod \"model-serving-api-6577596cd5-bmvnr\" (UID: \"fbcdca8b-8b39-46f0-ac72-64ac11529f65\") " pod="opendatahub/model-serving-api-6577596cd5-bmvnr" Jun 03 15:47:25.091058 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:25.090628 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-bmvnr" Jun 03 15:47:25.214786 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:25.214722 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-jwgq2" event={"ID":"57fd6338-2ffe-40bc-a9f4-5399b6bb441d","Type":"ContainerStarted","Data":"1c4dd5cf4dabcc9ce4c1d09122d7f3f4c7fc3f3460a289449d31524160ebec96"} Jun 03 15:47:25.246279 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:25.246241 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-bmvnr"] Jun 03 15:47:25.250274 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:47:25.250241 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbcdca8b_8b39_46f0_ac72_64ac11529f65.slice/crio-34aba5eaa323b58f47d14c1f49edc44fcda91a2fbb367239d8b7672cd53d831c WatchSource:0}: Error finding container 34aba5eaa323b58f47d14c1f49edc44fcda91a2fbb367239d8b7672cd53d831c: Status 404 returned error can't find the container with id 34aba5eaa323b58f47d14c1f49edc44fcda91a2fbb367239d8b7672cd53d831c Jun 03 15:47:26.218840 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:26.218798 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-bmvnr" event={"ID":"fbcdca8b-8b39-46f0-ac72-64ac11529f65","Type":"ContainerStarted","Data":"34aba5eaa323b58f47d14c1f49edc44fcda91a2fbb367239d8b7672cd53d831c"} Jun 03 15:47:29.233674 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:29.233627 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-jwgq2" event={"ID":"57fd6338-2ffe-40bc-a9f4-5399b6bb441d","Type":"ContainerStarted","Data":"70557efc691b8c5f2dd1e9a031665bfbb746c8ddfddbd68b643d18258a1d6ee7"} Jun 03 15:47:29.234154 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:29.233706 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-76588b95-jwgq2" Jun 03 15:47:29.235314 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:29.235286 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-bmvnr" event={"ID":"fbcdca8b-8b39-46f0-ac72-64ac11529f65","Type":"ContainerStarted","Data":"1831546bcb8619376c73965930009b61b729799af4fbf994b927a2e64ddc3735"} Jun 03 15:47:29.235534 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:29.235401 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-6577596cd5-bmvnr" Jun 03 15:47:29.253777 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:29.253722 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-76588b95-jwgq2" podStartSLOduration=1.981767401 podStartE2EDuration="6.25370381s" podCreationTimestamp="2026-06-03 15:47:23 +0000 UTC" firstStartedPulling="2026-06-03 15:47:24.422535161 +0000 UTC m=+391.807602357" lastFinishedPulling="2026-06-03 15:47:28.694471564 +0000 UTC m=+396.079538766" observedRunningTime="2026-06-03 15:47:29.252272902 +0000 UTC m=+396.637340122" watchObservedRunningTime="2026-06-03 15:47:29.25370381 +0000 UTC m=+396.638771031" Jun 03 15:47:29.269442 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:29.269383 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/model-serving-api-6577596cd5-bmvnr" podStartSLOduration=1.8167584639999999 podStartE2EDuration="5.269362815s" podCreationTimestamp="2026-06-03 15:47:24 +0000 UTC" firstStartedPulling="2026-06-03 15:47:25.252689813 +0000 UTC m=+392.637757025" lastFinishedPulling="2026-06-03 15:47:28.705294174 +0000 UTC m=+396.090361376" observedRunningTime="2026-06-03 15:47:29.268391888 +0000 UTC m=+396.653459105" watchObservedRunningTime="2026-06-03 15:47:29.269362815 +0000 UTC m=+396.654430032" Jun 03 15:47:32.000139 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.000096 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm"] Jun 03 15:47:32.002064 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.002047 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.004932 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.004908 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 03 15:47:32.005072 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.005053 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 03 15:47:32.006022 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.005995 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 03 15:47:32.006116 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.006053 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-q87fn\"" Jun 03 15:47:32.006116 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.006004 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 03 15:47:32.006116 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.006097 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 03 15:47:32.014354 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.014330 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm"] Jun 03 15:47:32.094714 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.094681 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/606b6b69-c604-4a32-a449-7178a7b1a866-cert\") pod \"lws-controller-manager-f96cfff67-7kvtm\" (UID: \"606b6b69-c604-4a32-a449-7178a7b1a866\") " pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.094902 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.094728 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/606b6b69-c604-4a32-a449-7178a7b1a866-manager-config\") pod \"lws-controller-manager-f96cfff67-7kvtm\" (UID: \"606b6b69-c604-4a32-a449-7178a7b1a866\") " pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.094902 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.094773 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bccd4\" (UniqueName: \"kubernetes.io/projected/606b6b69-c604-4a32-a449-7178a7b1a866-kube-api-access-bccd4\") pod \"lws-controller-manager-f96cfff67-7kvtm\" (UID: \"606b6b69-c604-4a32-a449-7178a7b1a866\") " pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.094902 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.094843 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/606b6b69-c604-4a32-a449-7178a7b1a866-metrics-cert\") pod \"lws-controller-manager-f96cfff67-7kvtm\" (UID: \"606b6b69-c604-4a32-a449-7178a7b1a866\") " pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.196306 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.196254 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/606b6b69-c604-4a32-a449-7178a7b1a866-metrics-cert\") pod \"lws-controller-manager-f96cfff67-7kvtm\" (UID: \"606b6b69-c604-4a32-a449-7178a7b1a866\") " pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.196469 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.196328 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/606b6b69-c604-4a32-a449-7178a7b1a866-cert\") pod \"lws-controller-manager-f96cfff67-7kvtm\" (UID: \"606b6b69-c604-4a32-a449-7178a7b1a866\") " pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.196469 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.196360 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/606b6b69-c604-4a32-a449-7178a7b1a866-manager-config\") pod \"lws-controller-manager-f96cfff67-7kvtm\" (UID: \"606b6b69-c604-4a32-a449-7178a7b1a866\") " pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.196469 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.196396 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bccd4\" (UniqueName: \"kubernetes.io/projected/606b6b69-c604-4a32-a449-7178a7b1a866-kube-api-access-bccd4\") pod \"lws-controller-manager-f96cfff67-7kvtm\" (UID: \"606b6b69-c604-4a32-a449-7178a7b1a866\") " pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.197017 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.196992 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/606b6b69-c604-4a32-a449-7178a7b1a866-manager-config\") pod \"lws-controller-manager-f96cfff67-7kvtm\" (UID: \"606b6b69-c604-4a32-a449-7178a7b1a866\") " pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.198741 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.198718 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/606b6b69-c604-4a32-a449-7178a7b1a866-cert\") pod \"lws-controller-manager-f96cfff67-7kvtm\" (UID: \"606b6b69-c604-4a32-a449-7178a7b1a866\") " pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.198741 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.198737 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/606b6b69-c604-4a32-a449-7178a7b1a866-metrics-cert\") pod \"lws-controller-manager-f96cfff67-7kvtm\" (UID: \"606b6b69-c604-4a32-a449-7178a7b1a866\") " pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.212673 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.212644 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bccd4\" (UniqueName: \"kubernetes.io/projected/606b6b69-c604-4a32-a449-7178a7b1a866-kube-api-access-bccd4\") pod \"lws-controller-manager-f96cfff67-7kvtm\" (UID: \"606b6b69-c604-4a32-a449-7178a7b1a866\") " pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.311026 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.310947 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:32.436664 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:32.436628 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm"] Jun 03 15:47:32.439745 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:47:32.439717 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod606b6b69_c604_4a32_a449_7178a7b1a866.slice/crio-ba3a4ecf98658db9e5390b79f7c6e17b5bb4cf51b9c486f5aff6ea20da961182 WatchSource:0}: Error finding container ba3a4ecf98658db9e5390b79f7c6e17b5bb4cf51b9c486f5aff6ea20da961182: Status 404 returned error can't find the container with id ba3a4ecf98658db9e5390b79f7c6e17b5bb4cf51b9c486f5aff6ea20da961182 Jun 03 15:47:33.246911 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:33.246878 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" event={"ID":"606b6b69-c604-4a32-a449-7178a7b1a866","Type":"ContainerStarted","Data":"ba3a4ecf98658db9e5390b79f7c6e17b5bb4cf51b9c486f5aff6ea20da961182"} Jun 03 15:47:35.253966 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:35.253938 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" event={"ID":"606b6b69-c604-4a32-a449-7178a7b1a866","Type":"ContainerStarted","Data":"178bc670fa8a3d91c58886b59ed0fab8af602983ab7e5150c7b9824bd1c9e851"} Jun 03 15:47:35.254365 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:35.254041 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:35.299126 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:35.299078 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" podStartSLOduration=1.559469615 podStartE2EDuration="4.29906357s" podCreationTimestamp="2026-06-03 15:47:31 +0000 UTC" firstStartedPulling="2026-06-03 15:47:32.441500393 +0000 UTC m=+399.826567589" lastFinishedPulling="2026-06-03 15:47:35.181094344 +0000 UTC m=+402.566161544" observedRunningTime="2026-06-03 15:47:35.298723994 +0000 UTC m=+402.683791213" watchObservedRunningTime="2026-06-03 15:47:35.29906357 +0000 UTC m=+402.684130789" Jun 03 15:47:40.240538 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:40.240504 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-76588b95-jwgq2" Jun 03 15:47:40.242459 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:40.242434 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-6577596cd5-bmvnr" Jun 03 15:47:46.258845 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:46.258810 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-f96cfff67-7kvtm" Jun 03 15:47:49.316724 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.316688 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-7888f9cd56-pgr29"] Jun 03 15:47:49.318574 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.318557 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7888f9cd56-pgr29" Jun 03 15:47:49.320875 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.320851 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 03 15:47:49.320991 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.320917 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5b4vv\"" Jun 03 15:47:49.328462 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.328442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/bd517f90-303b-421d-9c50-7b079cd30f25-redirect-config\") pod \"dashboard-redirect-7888f9cd56-pgr29\" (UID: \"bd517f90-303b-421d-9c50-7b079cd30f25\") " pod="opendatahub/dashboard-redirect-7888f9cd56-pgr29" Jun 03 15:47:49.328563 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.328495 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nrls\" (UniqueName: \"kubernetes.io/projected/bd517f90-303b-421d-9c50-7b079cd30f25-kube-api-access-8nrls\") pod \"dashboard-redirect-7888f9cd56-pgr29\" (UID: \"bd517f90-303b-421d-9c50-7b079cd30f25\") " pod="opendatahub/dashboard-redirect-7888f9cd56-pgr29" Jun 03 15:47:49.331931 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.331907 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7888f9cd56-pgr29"] Jun 03 15:47:49.429472 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.429432 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8nrls\" (UniqueName: \"kubernetes.io/projected/bd517f90-303b-421d-9c50-7b079cd30f25-kube-api-access-8nrls\") pod \"dashboard-redirect-7888f9cd56-pgr29\" (UID: \"bd517f90-303b-421d-9c50-7b079cd30f25\") " pod="opendatahub/dashboard-redirect-7888f9cd56-pgr29" Jun 03 15:47:49.429645 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.429500 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/bd517f90-303b-421d-9c50-7b079cd30f25-redirect-config\") pod \"dashboard-redirect-7888f9cd56-pgr29\" (UID: \"bd517f90-303b-421d-9c50-7b079cd30f25\") " pod="opendatahub/dashboard-redirect-7888f9cd56-pgr29" Jun 03 15:47:49.430322 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.430299 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/bd517f90-303b-421d-9c50-7b079cd30f25-redirect-config\") pod \"dashboard-redirect-7888f9cd56-pgr29\" (UID: \"bd517f90-303b-421d-9c50-7b079cd30f25\") " pod="opendatahub/dashboard-redirect-7888f9cd56-pgr29" Jun 03 15:47:49.437493 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.437468 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nrls\" (UniqueName: \"kubernetes.io/projected/bd517f90-303b-421d-9c50-7b079cd30f25-kube-api-access-8nrls\") pod \"dashboard-redirect-7888f9cd56-pgr29\" (UID: \"bd517f90-303b-421d-9c50-7b079cd30f25\") " pod="opendatahub/dashboard-redirect-7888f9cd56-pgr29" Jun 03 15:47:49.627660 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.627576 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7888f9cd56-pgr29" Jun 03 15:47:49.770234 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:49.770181 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7888f9cd56-pgr29"] Jun 03 15:47:49.773141 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:47:49.773114 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd517f90_303b_421d_9c50_7b079cd30f25.slice/crio-12f91562cc0c7069b09db6c6e4b412fcaf963c8cad3ed4c656dd1802d90bae24 WatchSource:0}: Error finding container 12f91562cc0c7069b09db6c6e4b412fcaf963c8cad3ed4c656dd1802d90bae24: Status 404 returned error can't find the container with id 12f91562cc0c7069b09db6c6e4b412fcaf963c8cad3ed4c656dd1802d90bae24 Jun 03 15:47:50.299186 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:50.299151 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7888f9cd56-pgr29" event={"ID":"bd517f90-303b-421d-9c50-7b079cd30f25","Type":"ContainerStarted","Data":"12f91562cc0c7069b09db6c6e4b412fcaf963c8cad3ed4c656dd1802d90bae24"} Jun 03 15:47:55.316349 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:47:55.316308 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7888f9cd56-pgr29" event={"ID":"bd517f90-303b-421d-9c50-7b079cd30f25","Type":"ContainerStarted","Data":"72eaccf6552ab9b07b288f2563bf870a7f5133184cea20b3d989103647c0f7fe"} Jun 03 15:48:05.885666 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.885543 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-7888f9cd56-pgr29" podStartSLOduration=11.695403833 podStartE2EDuration="16.885523783s" podCreationTimestamp="2026-06-03 15:47:49 +0000 UTC" firstStartedPulling="2026-06-03 15:47:49.774498381 +0000 UTC m=+417.159565579" lastFinishedPulling="2026-06-03 15:47:54.964618314 +0000 UTC m=+422.349685529" observedRunningTime="2026-06-03 15:47:55.340844805 +0000 UTC m=+422.725912023" watchObservedRunningTime="2026-06-03 15:48:05.885523783 +0000 UTC m=+433.270591003" Jun 03 15:48:05.886102 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.885713 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb"] Jun 03 15:48:05.887828 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.887810 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:05.890422 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.890401 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 03 15:48:05.890523 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.890401 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 03 15:48:05.890523 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.890401 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-d86pk\"" Jun 03 15:48:05.890618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.890404 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 03 15:48:05.899590 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.899565 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb"] Jun 03 15:48:05.958903 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.958864 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:05.958903 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.958910 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:05.959127 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.958932 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:05.959127 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.958959 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:05.959127 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.958983 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:05.959127 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.959004 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:05.959127 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.959025 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn672\" (UniqueName: \"kubernetes.io/projected/8184d334-ba43-408f-84b7-6ac5c8d9091d-kube-api-access-qn672\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:05.959127 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.959059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8184d334-ba43-408f-84b7-6ac5c8d9091d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:05.959127 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:05.959085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.060454 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.060406 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.060454 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.060460 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.060678 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.060489 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.060678 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.060518 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.060678 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.060553 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.060678 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.060587 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.060678 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.060611 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qn672\" (UniqueName: \"kubernetes.io/projected/8184d334-ba43-408f-84b7-6ac5c8d9091d-kube-api-access-qn672\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.060678 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.060643 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8184d334-ba43-408f-84b7-6ac5c8d9091d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.060678 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.060675 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.061031 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.060892 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.061031 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.060963 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.061096 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.061026 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.061096 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.061088 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.061494 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.061471 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8184d334-ba43-408f-84b7-6ac5c8d9091d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.062967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.062947 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.063055 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.062994 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.068861 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.068829 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.068972 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.068861 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn672\" (UniqueName: \"kubernetes.io/projected/8184d334-ba43-408f-84b7-6ac5c8d9091d-kube-api-access-qn672\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.125547 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.125516 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k"] Jun 03 15:48:06.127988 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.127967 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.143128 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.143062 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k"] Jun 03 15:48:06.161152 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.161108 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.161359 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.161169 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-977pq\" (UniqueName: \"kubernetes.io/projected/7232360c-879b-4995-a402-c860535d44dc-kube-api-access-977pq\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.161359 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.161266 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.161359 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.161302 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7232360c-879b-4995-a402-c860535d44dc-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.161359 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.161336 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.161553 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.161378 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7232360c-879b-4995-a402-c860535d44dc-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.161553 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.161431 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.161553 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.161482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.161553 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.161509 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7232360c-879b-4995-a402-c860535d44dc-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.197423 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.197386 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:06.262293 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.262252 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7232360c-879b-4995-a402-c860535d44dc-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.262434 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.262295 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.262434 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.262327 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7232360c-879b-4995-a402-c860535d44dc-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.262434 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.262361 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.262434 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.262404 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.262434 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.262432 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7232360c-879b-4995-a402-c860535d44dc-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.262593 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.262463 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.262593 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.262497 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-977pq\" (UniqueName: \"kubernetes.io/projected/7232360c-879b-4995-a402-c860535d44dc-kube-api-access-977pq\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.262593 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.262549 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.263121 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.262787 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.263121 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.263060 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.263343 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.263120 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.263343 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.263138 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.263343 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.263146 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7232360c-879b-4995-a402-c860535d44dc-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.264967 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.264947 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7232360c-879b-4995-a402-c860535d44dc-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.265177 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.265161 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7232360c-879b-4995-a402-c860535d44dc-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.281085 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.281037 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-977pq\" (UniqueName: \"kubernetes.io/projected/7232360c-879b-4995-a402-c860535d44dc-kube-api-access-977pq\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.297250 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.297192 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7232360c-879b-4995-a402-c860535d44dc-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjx92k\" (UID: \"7232360c-879b-4995-a402-c860535d44dc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.337772 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.337730 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb"] Jun 03 15:48:06.339952 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:48:06.339925 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8184d334_ba43_408f_84b7_6ac5c8d9091d.slice/crio-80a04c0920a2ebe23d850edec26cf2ae2b151e0de5252fa62cc2373da92ab404 WatchSource:0}: Error finding container 80a04c0920a2ebe23d850edec26cf2ae2b151e0de5252fa62cc2373da92ab404: Status 404 returned error can't find the container with id 80a04c0920a2ebe23d850edec26cf2ae2b151e0de5252fa62cc2373da92ab404 Jun 03 15:48:06.347145 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.347108 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" event={"ID":"8184d334-ba43-408f-84b7-6ac5c8d9091d","Type":"ContainerStarted","Data":"80a04c0920a2ebe23d850edec26cf2ae2b151e0de5252fa62cc2373da92ab404"} Jun 03 15:48:06.438796 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.438696 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:06.571185 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:06.571163 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k"] Jun 03 15:48:06.572659 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:48:06.572631 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7232360c_879b_4995_a402_c860535d44dc.slice/crio-301f93066eb42328b4c9d57ae2ff0b3057568ac13ce5e9c45c00324d97b731de WatchSource:0}: Error finding container 301f93066eb42328b4c9d57ae2ff0b3057568ac13ce5e9c45c00324d97b731de: Status 404 returned error can't find the container with id 301f93066eb42328b4c9d57ae2ff0b3057568ac13ce5e9c45c00324d97b731de Jun 03 15:48:07.357150 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:07.357108 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" event={"ID":"7232360c-879b-4995-a402-c860535d44dc","Type":"ContainerStarted","Data":"301f93066eb42328b4c9d57ae2ff0b3057568ac13ce5e9c45c00324d97b731de"} Jun 03 15:48:09.245033 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:09.244992 2578 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 03 15:48:09.245339 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:09.245071 2578 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 03 15:48:09.245339 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:09.245118 2578 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 03 15:48:09.249630 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:09.249606 2578 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 03 15:48:09.249692 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:09.249662 2578 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 03 15:48:09.249692 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:09.249688 2578 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 03 15:48:09.369720 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:09.369670 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" event={"ID":"8184d334-ba43-408f-84b7-6ac5c8d9091d","Type":"ContainerStarted","Data":"2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97"} Jun 03 15:48:09.371365 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:09.371333 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" event={"ID":"7232360c-879b-4995-a402-c860535d44dc","Type":"ContainerStarted","Data":"779c1c4610d999016bbd3ee4d606488aa32bbba6e09d59d9997a175b72e1c4c7"} Jun 03 15:48:09.396137 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:09.394291 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" podStartSLOduration=1.491355167 podStartE2EDuration="4.394273611s" podCreationTimestamp="2026-06-03 15:48:05 +0000 UTC" firstStartedPulling="2026-06-03 15:48:06.341820705 +0000 UTC m=+433.726887919" lastFinishedPulling="2026-06-03 15:48:09.244739162 +0000 UTC m=+436.629806363" observedRunningTime="2026-06-03 15:48:09.392245845 +0000 UTC m=+436.777313065" watchObservedRunningTime="2026-06-03 15:48:09.394273611 +0000 UTC m=+436.779340831" Jun 03 15:48:09.418618 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:09.418557 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" podStartSLOduration=0.74365342 podStartE2EDuration="3.418542488s" podCreationTimestamp="2026-06-03 15:48:06 +0000 UTC" firstStartedPulling="2026-06-03 15:48:06.574519965 +0000 UTC m=+433.959587161" lastFinishedPulling="2026-06-03 15:48:09.249409021 +0000 UTC m=+436.634476229" observedRunningTime="2026-06-03 15:48:09.417086274 +0000 UTC m=+436.802153494" watchObservedRunningTime="2026-06-03 15:48:09.418542488 +0000 UTC m=+436.803609707" Jun 03 15:48:09.439360 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:09.439329 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:10.198395 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:10.198354 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:10.199573 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:10.199547 2578 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.18:15021/healthz/ready\": dial tcp 10.134.0.18:15021: connect: connection refused" start-of-body= Jun 03 15:48:10.199667 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:10.199594 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" podUID="8184d334-ba43-408f-84b7-6ac5c8d9091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.18:15021/healthz/ready\": dial tcp 10.134.0.18:15021: connect: connection refused" Jun 03 15:48:10.443771 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:10.443737 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:11.198596 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:11.198553 2578 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.18:15021/healthz/ready\": dial tcp 10.134.0.18:15021: connect: connection refused" start-of-body= Jun 03 15:48:11.198776 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:11.198620 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" podUID="8184d334-ba43-408f-84b7-6ac5c8d9091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.18:15021/healthz/ready\": dial tcp 10.134.0.18:15021: connect: connection refused" Jun 03 15:48:11.377428 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:11.377397 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:11.378345 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:11.378317 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjx92k" Jun 03 15:48:11.434329 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:11.434297 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb"] Jun 03 15:48:12.198566 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:12.198530 2578 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.18:15021/healthz/ready\": dial tcp 10.134.0.18:15021: connect: connection refused" start-of-body= Jun 03 15:48:12.198957 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:12.198586 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" podUID="8184d334-ba43-408f-84b7-6ac5c8d9091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.18:15021/healthz/ready\": dial tcp 10.134.0.18:15021: connect: connection refused" Jun 03 15:48:12.380301 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:12.380248 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" podUID="8184d334-ba43-408f-84b7-6ac5c8d9091d" containerName="istio-proxy" containerID="cri-o://2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97" gracePeriod=30 Jun 03 15:48:17.628444 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.628420 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:17.762296 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762260 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-credential-socket\") pod \"8184d334-ba43-408f-84b7-6ac5c8d9091d\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " Jun 03 15:48:17.762479 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762301 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-podinfo\") pod \"8184d334-ba43-408f-84b7-6ac5c8d9091d\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " Jun 03 15:48:17.762479 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762349 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-token\") pod \"8184d334-ba43-408f-84b7-6ac5c8d9091d\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " Jun 03 15:48:17.762479 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762369 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-workload-socket\") pod \"8184d334-ba43-408f-84b7-6ac5c8d9091d\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " Jun 03 15:48:17.762479 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762415 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-data\") pod \"8184d334-ba43-408f-84b7-6ac5c8d9091d\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " Jun 03 15:48:17.762479 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762434 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-envoy\") pod \"8184d334-ba43-408f-84b7-6ac5c8d9091d\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " Jun 03 15:48:17.762479 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762461 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-workload-certs\") pod \"8184d334-ba43-408f-84b7-6ac5c8d9091d\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " Jun 03 15:48:17.762777 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762482 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qn672\" (UniqueName: \"kubernetes.io/projected/8184d334-ba43-408f-84b7-6ac5c8d9091d-kube-api-access-qn672\") pod \"8184d334-ba43-408f-84b7-6ac5c8d9091d\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " Jun 03 15:48:17.762777 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762516 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8184d334-ba43-408f-84b7-6ac5c8d9091d-istiod-ca-cert\") pod \"8184d334-ba43-408f-84b7-6ac5c8d9091d\" (UID: \"8184d334-ba43-408f-84b7-6ac5c8d9091d\") " Jun 03 15:48:17.762777 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762659 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8184d334-ba43-408f-84b7-6ac5c8d9091d" (UID: "8184d334-ba43-408f-84b7-6ac5c8d9091d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 15:48:17.762777 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762688 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8184d334-ba43-408f-84b7-6ac5c8d9091d" (UID: "8184d334-ba43-408f-84b7-6ac5c8d9091d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 15:48:17.762948 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762812 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-credential-socket\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:48:17.762948 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762832 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-workload-socket\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:48:17.762948 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762831 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8184d334-ba43-408f-84b7-6ac5c8d9091d" (UID: "8184d334-ba43-408f-84b7-6ac5c8d9091d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 15:48:17.763086 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.762965 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-data" (OuterVolumeSpecName: "istio-data") pod "8184d334-ba43-408f-84b7-6ac5c8d9091d" (UID: "8184d334-ba43-408f-84b7-6ac5c8d9091d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 15:48:17.763136 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.763089 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8184d334-ba43-408f-84b7-6ac5c8d9091d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8184d334-ba43-408f-84b7-6ac5c8d9091d" (UID: "8184d334-ba43-408f-84b7-6ac5c8d9091d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 15:48:17.764870 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.764843 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8184d334-ba43-408f-84b7-6ac5c8d9091d" (UID: "8184d334-ba43-408f-84b7-6ac5c8d9091d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 15:48:17.764997 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.764927 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8184d334-ba43-408f-84b7-6ac5c8d9091d-kube-api-access-qn672" (OuterVolumeSpecName: "kube-api-access-qn672") pod "8184d334-ba43-408f-84b7-6ac5c8d9091d" (UID: "8184d334-ba43-408f-84b7-6ac5c8d9091d"). InnerVolumeSpecName "kube-api-access-qn672". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 15:48:17.765058 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.765002 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-token" (OuterVolumeSpecName: "istio-token") pod "8184d334-ba43-408f-84b7-6ac5c8d9091d" (UID: "8184d334-ba43-408f-84b7-6ac5c8d9091d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 15:48:17.765058 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.765008 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8184d334-ba43-408f-84b7-6ac5c8d9091d" (UID: "8184d334-ba43-408f-84b7-6ac5c8d9091d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 03 15:48:17.863604 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.863559 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-data\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:48:17.863604 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.863597 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-envoy\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:48:17.863604 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.863609 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8184d334-ba43-408f-84b7-6ac5c8d9091d-workload-certs\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:48:17.863837 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.863623 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qn672\" (UniqueName: \"kubernetes.io/projected/8184d334-ba43-408f-84b7-6ac5c8d9091d-kube-api-access-qn672\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:48:17.863837 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.863636 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8184d334-ba43-408f-84b7-6ac5c8d9091d-istiod-ca-cert\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:48:17.863837 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.863649 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-podinfo\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:48:17.863837 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:17.863659 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8184d334-ba43-408f-84b7-6ac5c8d9091d-istio-token\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:48:18.400435 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:18.400397 2578 generic.go:358] "Generic (PLEG): container finished" podID="8184d334-ba43-408f-84b7-6ac5c8d9091d" containerID="2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97" exitCode=0 Jun 03 15:48:18.400637 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:18.400446 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" event={"ID":"8184d334-ba43-408f-84b7-6ac5c8d9091d","Type":"ContainerDied","Data":"2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97"} Jun 03 15:48:18.400637 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:18.400457 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" Jun 03 15:48:18.400637 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:18.400474 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb" event={"ID":"8184d334-ba43-408f-84b7-6ac5c8d9091d","Type":"ContainerDied","Data":"80a04c0920a2ebe23d850edec26cf2ae2b151e0de5252fa62cc2373da92ab404"} Jun 03 15:48:18.400637 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:18.400494 2578 scope.go:117] "RemoveContainer" containerID="2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97" Jun 03 15:48:18.408879 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:18.408857 2578 scope.go:117] "RemoveContainer" containerID="2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97" Jun 03 15:48:18.409177 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:48:18.409155 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97\": container with ID starting with 2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97 not found: ID does not exist" containerID="2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97" Jun 03 15:48:18.409243 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:18.409187 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97"} err="failed to get container status \"2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97\": rpc error: code = NotFound desc = could not find container \"2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97\": container with ID starting with 2fa0eb40a307e13a38323a41780f46ee567317735051a4d74499fccc18d6aa97 not found: ID does not exist" Jun 03 15:48:18.423390 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:18.423359 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb"] Jun 03 15:48:18.427542 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:18.427522 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44xq9qb"] Jun 03 15:48:19.111721 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:19.111675 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8184d334-ba43-408f-84b7-6ac5c8d9091d" path="/var/lib/kubelet/pods/8184d334-ba43-408f-84b7-6ac5c8d9091d/volumes" Jun 03 15:48:24.254724 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.254687 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4jpx2"] Jun 03 15:48:24.255107 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.254987 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8184d334-ba43-408f-84b7-6ac5c8d9091d" containerName="istio-proxy" Jun 03 15:48:24.255107 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.255000 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8184d334-ba43-408f-84b7-6ac5c8d9091d" containerName="istio-proxy" Jun 03 15:48:24.255107 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.255042 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="8184d334-ba43-408f-84b7-6ac5c8d9091d" containerName="istio-proxy" Jun 03 15:48:24.297393 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.297352 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4jpx2"] Jun 03 15:48:24.297560 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.297480 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" Jun 03 15:48:24.300073 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.300026 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 15:48:24.300249 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.300047 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-c4qmf\"" Jun 03 15:48:24.301263 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.301244 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 15:48:24.419304 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.419275 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp6j7\" (UniqueName: \"kubernetes.io/projected/2ed8001c-ca8d-48a2-bf3a-09574183e15a-kube-api-access-bp6j7\") pod \"kuadrant-operator-catalog-4jpx2\" (UID: \"2ed8001c-ca8d-48a2-bf3a-09574183e15a\") " pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" Jun 03 15:48:24.520439 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.520345 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bp6j7\" (UniqueName: \"kubernetes.io/projected/2ed8001c-ca8d-48a2-bf3a-09574183e15a-kube-api-access-bp6j7\") pod \"kuadrant-operator-catalog-4jpx2\" (UID: \"2ed8001c-ca8d-48a2-bf3a-09574183e15a\") " pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" Jun 03 15:48:24.529434 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.529405 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp6j7\" (UniqueName: \"kubernetes.io/projected/2ed8001c-ca8d-48a2-bf3a-09574183e15a-kube-api-access-bp6j7\") pod \"kuadrant-operator-catalog-4jpx2\" (UID: \"2ed8001c-ca8d-48a2-bf3a-09574183e15a\") " pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" Jun 03 15:48:24.606339 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.606278 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" Jun 03 15:48:24.613566 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.613536 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4jpx2"] Jun 03 15:48:24.734575 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.734501 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4jpx2"] Jun 03 15:48:24.736709 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:48:24.736684 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed8001c_ca8d_48a2_bf3a_09574183e15a.slice/crio-92474d2056d4256ba3b1d75fb2e29a2281872c97e99accee11f9bd9b6acd4125 WatchSource:0}: Error finding container 92474d2056d4256ba3b1d75fb2e29a2281872c97e99accee11f9bd9b6acd4125: Status 404 returned error can't find the container with id 92474d2056d4256ba3b1d75fb2e29a2281872c97e99accee11f9bd9b6acd4125 Jun 03 15:48:24.825411 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.825324 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9vstf"] Jun 03 15:48:24.911457 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.911420 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9vstf"] Jun 03 15:48:24.911608 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:24.911539 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9vstf" Jun 03 15:48:25.024897 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:25.024859 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fwsj\" (UniqueName: \"kubernetes.io/projected/5b94c742-a5b1-4870-8d94-bab39a7902cc-kube-api-access-9fwsj\") pod \"kuadrant-operator-catalog-9vstf\" (UID: \"5b94c742-a5b1-4870-8d94-bab39a7902cc\") " pod="kuadrant-system/kuadrant-operator-catalog-9vstf" Jun 03 15:48:25.125661 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:25.125578 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9fwsj\" (UniqueName: \"kubernetes.io/projected/5b94c742-a5b1-4870-8d94-bab39a7902cc-kube-api-access-9fwsj\") pod \"kuadrant-operator-catalog-9vstf\" (UID: \"5b94c742-a5b1-4870-8d94-bab39a7902cc\") " pod="kuadrant-system/kuadrant-operator-catalog-9vstf" Jun 03 15:48:25.135465 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:25.135435 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fwsj\" (UniqueName: \"kubernetes.io/projected/5b94c742-a5b1-4870-8d94-bab39a7902cc-kube-api-access-9fwsj\") pod \"kuadrant-operator-catalog-9vstf\" (UID: \"5b94c742-a5b1-4870-8d94-bab39a7902cc\") " pod="kuadrant-system/kuadrant-operator-catalog-9vstf" Jun 03 15:48:25.221061 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:25.220983 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9vstf" Jun 03 15:48:25.353048 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:25.353023 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9vstf"] Jun 03 15:48:25.355428 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:48:25.355399 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b94c742_a5b1_4870_8d94_bab39a7902cc.slice/crio-7eebc3c90a9811508c8efa35c4cc74a87112c3e248682af3966a8b84d70efc41 WatchSource:0}: Error finding container 7eebc3c90a9811508c8efa35c4cc74a87112c3e248682af3966a8b84d70efc41: Status 404 returned error can't find the container with id 7eebc3c90a9811508c8efa35c4cc74a87112c3e248682af3966a8b84d70efc41 Jun 03 15:48:25.421446 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:25.421367 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9vstf" event={"ID":"5b94c742-a5b1-4870-8d94-bab39a7902cc","Type":"ContainerStarted","Data":"7eebc3c90a9811508c8efa35c4cc74a87112c3e248682af3966a8b84d70efc41"} Jun 03 15:48:25.422354 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:25.422321 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" event={"ID":"2ed8001c-ca8d-48a2-bf3a-09574183e15a","Type":"ContainerStarted","Data":"92474d2056d4256ba3b1d75fb2e29a2281872c97e99accee11f9bd9b6acd4125"} Jun 03 15:48:27.433543 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:27.433506 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9vstf" event={"ID":"5b94c742-a5b1-4870-8d94-bab39a7902cc","Type":"ContainerStarted","Data":"669d1e7ca87999d3a3196e5a5ccf63b980d27e3b425eb2ffdf114206a0b3c0e1"} Jun 03 15:48:27.435142 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:27.435109 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" event={"ID":"2ed8001c-ca8d-48a2-bf3a-09574183e15a","Type":"ContainerStarted","Data":"d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340"} Jun 03 15:48:27.435267 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:27.435231 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" podUID="2ed8001c-ca8d-48a2-bf3a-09574183e15a" containerName="registry-server" containerID="cri-o://d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340" gracePeriod=2 Jun 03 15:48:27.450601 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:27.450419 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-9vstf" podStartSLOduration=1.479043044 podStartE2EDuration="3.450400472s" podCreationTimestamp="2026-06-03 15:48:24 +0000 UTC" firstStartedPulling="2026-06-03 15:48:25.356816606 +0000 UTC m=+452.741883803" lastFinishedPulling="2026-06-03 15:48:27.328174031 +0000 UTC m=+454.713241231" observedRunningTime="2026-06-03 15:48:27.449932692 +0000 UTC m=+454.834999925" watchObservedRunningTime="2026-06-03 15:48:27.450400472 +0000 UTC m=+454.835467691" Jun 03 15:48:27.466819 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:27.466479 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" podStartSLOduration=0.880412766 podStartE2EDuration="3.466461576s" podCreationTimestamp="2026-06-03 15:48:24 +0000 UTC" firstStartedPulling="2026-06-03 15:48:24.738326517 +0000 UTC m=+452.123393728" lastFinishedPulling="2026-06-03 15:48:27.32437533 +0000 UTC m=+454.709442538" observedRunningTime="2026-06-03 15:48:27.46572642 +0000 UTC m=+454.850793639" watchObservedRunningTime="2026-06-03 15:48:27.466461576 +0000 UTC m=+454.851528796" Jun 03 15:48:27.773143 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:27.773119 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" Jun 03 15:48:27.851977 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:27.851932 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp6j7\" (UniqueName: \"kubernetes.io/projected/2ed8001c-ca8d-48a2-bf3a-09574183e15a-kube-api-access-bp6j7\") pod \"2ed8001c-ca8d-48a2-bf3a-09574183e15a\" (UID: \"2ed8001c-ca8d-48a2-bf3a-09574183e15a\") " Jun 03 15:48:27.854077 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:27.854055 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ed8001c-ca8d-48a2-bf3a-09574183e15a-kube-api-access-bp6j7" (OuterVolumeSpecName: "kube-api-access-bp6j7") pod "2ed8001c-ca8d-48a2-bf3a-09574183e15a" (UID: "2ed8001c-ca8d-48a2-bf3a-09574183e15a"). InnerVolumeSpecName "kube-api-access-bp6j7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 15:48:27.953508 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:27.953414 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp6j7\" (UniqueName: \"kubernetes.io/projected/2ed8001c-ca8d-48a2-bf3a-09574183e15a-kube-api-access-bp6j7\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:48:28.439572 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:28.439534 2578 generic.go:358] "Generic (PLEG): container finished" podID="2ed8001c-ca8d-48a2-bf3a-09574183e15a" containerID="d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340" exitCode=0 Jun 03 15:48:28.440025 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:28.439592 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" Jun 03 15:48:28.440025 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:28.439618 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" event={"ID":"2ed8001c-ca8d-48a2-bf3a-09574183e15a","Type":"ContainerDied","Data":"d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340"} Jun 03 15:48:28.440025 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:28.439652 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4jpx2" event={"ID":"2ed8001c-ca8d-48a2-bf3a-09574183e15a","Type":"ContainerDied","Data":"92474d2056d4256ba3b1d75fb2e29a2281872c97e99accee11f9bd9b6acd4125"} Jun 03 15:48:28.440025 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:28.439672 2578 scope.go:117] "RemoveContainer" containerID="d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340" Jun 03 15:48:28.447746 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:28.447730 2578 scope.go:117] "RemoveContainer" containerID="d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340" Jun 03 15:48:28.447967 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:48:28.447946 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340\": container with ID starting with d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340 not found: ID does not exist" containerID="d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340" Jun 03 15:48:28.448018 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:28.447974 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340"} err="failed to get container status \"d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340\": rpc error: code = NotFound desc = could not find container \"d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340\": container with ID starting with d2570b7ef368c87c4c3406de4e144fd42b42efbb704009c5f04e9369c7f80340 not found: ID does not exist" Jun 03 15:48:28.480870 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:28.480841 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4jpx2"] Jun 03 15:48:28.485519 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:28.485493 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4jpx2"] Jun 03 15:48:29.115460 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:29.115427 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ed8001c-ca8d-48a2-bf3a-09574183e15a" path="/var/lib/kubelet/pods/2ed8001c-ca8d-48a2-bf3a-09574183e15a/volumes" Jun 03 15:48:35.221248 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:35.221194 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-9vstf" Jun 03 15:48:35.221248 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:35.221253 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-9vstf" Jun 03 15:48:35.243181 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:35.243153 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-9vstf" Jun 03 15:48:35.482906 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:35.482830 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-9vstf" Jun 03 15:48:56.199160 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.199122 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll"] Jun 03 15:48:56.199538 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.199491 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ed8001c-ca8d-48a2-bf3a-09574183e15a" containerName="registry-server" Jun 03 15:48:56.199538 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.199505 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ed8001c-ca8d-48a2-bf3a-09574183e15a" containerName="registry-server" Jun 03 15:48:56.199609 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.199550 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="2ed8001c-ca8d-48a2-bf3a-09574183e15a" containerName="registry-server" Jun 03 15:48:56.208460 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.208435 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll" Jun 03 15:48:56.211114 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.211089 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 03 15:48:56.211262 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.211168 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-t8pkg\"" Jun 03 15:48:56.213765 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.213744 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll"] Jun 03 15:48:56.274877 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.274842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4szbn\" (UniqueName: \"kubernetes.io/projected/1d5b300f-6c7e-4076-83e1-28ced13cf5b1-kube-api-access-4szbn\") pod \"dns-operator-controller-manager-648d5c98bc-zn9ll\" (UID: \"1d5b300f-6c7e-4076-83e1-28ced13cf5b1\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll" Jun 03 15:48:56.375243 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.375188 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4szbn\" (UniqueName: \"kubernetes.io/projected/1d5b300f-6c7e-4076-83e1-28ced13cf5b1-kube-api-access-4szbn\") pod \"dns-operator-controller-manager-648d5c98bc-zn9ll\" (UID: \"1d5b300f-6c7e-4076-83e1-28ced13cf5b1\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll" Jun 03 15:48:56.391926 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.391895 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4szbn\" (UniqueName: \"kubernetes.io/projected/1d5b300f-6c7e-4076-83e1-28ced13cf5b1-kube-api-access-4szbn\") pod \"dns-operator-controller-manager-648d5c98bc-zn9ll\" (UID: \"1d5b300f-6c7e-4076-83e1-28ced13cf5b1\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll" Jun 03 15:48:56.520112 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.520071 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll" Jun 03 15:48:56.658111 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:56.658079 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll"] Jun 03 15:48:56.660577 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:48:56.660541 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d5b300f_6c7e_4076_83e1_28ced13cf5b1.slice/crio-3e9b1a863740d85845b7b56b094adcac23fa3446ab13853803782422a4d1a287 WatchSource:0}: Error finding container 3e9b1a863740d85845b7b56b094adcac23fa3446ab13853803782422a4d1a287: Status 404 returned error can't find the container with id 3e9b1a863740d85845b7b56b094adcac23fa3446ab13853803782422a4d1a287 Jun 03 15:48:57.530677 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:57.530631 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll" event={"ID":"1d5b300f-6c7e-4076-83e1-28ced13cf5b1","Type":"ContainerStarted","Data":"3e9b1a863740d85845b7b56b094adcac23fa3446ab13853803782422a4d1a287"} Jun 03 15:48:59.538378 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:59.538336 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll" event={"ID":"1d5b300f-6c7e-4076-83e1-28ced13cf5b1","Type":"ContainerStarted","Data":"31ea0656786badf78a43b54b55ac9278e3eaf735213a68c9caa461602f68266e"} Jun 03 15:48:59.538788 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:59.538487 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll" Jun 03 15:48:59.564311 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:59.564263 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll" podStartSLOduration=1.071015657 podStartE2EDuration="3.564250177s" podCreationTimestamp="2026-06-03 15:48:56 +0000 UTC" firstStartedPulling="2026-06-03 15:48:56.66492836 +0000 UTC m=+484.049995558" lastFinishedPulling="2026-06-03 15:48:59.158162865 +0000 UTC m=+486.543230078" observedRunningTime="2026-06-03 15:48:59.562885055 +0000 UTC m=+486.947952274" watchObservedRunningTime="2026-06-03 15:48:59.564250177 +0000 UTC m=+486.949317395" Jun 03 15:48:59.664684 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:59.664645 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4"] Jun 03 15:48:59.667740 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:59.667721 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" Jun 03 15:48:59.672018 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:59.671996 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-45jzn\"" Jun 03 15:48:59.684837 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:59.684808 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4"] Jun 03 15:48:59.701131 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:59.701102 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs7l2\" (UniqueName: \"kubernetes.io/projected/efe0b195-6563-4a69-9e36-f18c5edaaac0-kube-api-access-vs7l2\") pod \"limitador-operator-controller-manager-85c4996f8c-8rsv4\" (UID: \"efe0b195-6563-4a69-9e36-f18c5edaaac0\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" Jun 03 15:48:59.802512 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:59.802418 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vs7l2\" (UniqueName: \"kubernetes.io/projected/efe0b195-6563-4a69-9e36-f18c5edaaac0-kube-api-access-vs7l2\") pod \"limitador-operator-controller-manager-85c4996f8c-8rsv4\" (UID: \"efe0b195-6563-4a69-9e36-f18c5edaaac0\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" Jun 03 15:48:59.830939 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:59.830914 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs7l2\" (UniqueName: \"kubernetes.io/projected/efe0b195-6563-4a69-9e36-f18c5edaaac0-kube-api-access-vs7l2\") pod \"limitador-operator-controller-manager-85c4996f8c-8rsv4\" (UID: \"efe0b195-6563-4a69-9e36-f18c5edaaac0\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" Jun 03 15:48:59.977907 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:48:59.977876 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" Jun 03 15:49:00.114638 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:00.114612 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4"] Jun 03 15:49:00.117056 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:49:00.117025 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe0b195_6563_4a69_9e36_f18c5edaaac0.slice/crio-25af01928492a6a8d25f31b05eca44f8e27887f66d039d4b05430feca998df50 WatchSource:0}: Error finding container 25af01928492a6a8d25f31b05eca44f8e27887f66d039d4b05430feca998df50: Status 404 returned error can't find the container with id 25af01928492a6a8d25f31b05eca44f8e27887f66d039d4b05430feca998df50 Jun 03 15:49:00.542732 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:00.542692 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" event={"ID":"efe0b195-6563-4a69-9e36-f18c5edaaac0","Type":"ContainerStarted","Data":"25af01928492a6a8d25f31b05eca44f8e27887f66d039d4b05430feca998df50"} Jun 03 15:49:02.550540 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:02.550455 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" event={"ID":"efe0b195-6563-4a69-9e36-f18c5edaaac0","Type":"ContainerStarted","Data":"576ae745b9fc23ebbdb3c9fbaa24c017a2148470ebeac4d0ea45b8fa96de57eb"} Jun 03 15:49:02.550903 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:02.550553 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" Jun 03 15:49:02.580039 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:02.579986 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" podStartSLOduration=1.458348946 podStartE2EDuration="3.579970202s" podCreationTimestamp="2026-06-03 15:48:59 +0000 UTC" firstStartedPulling="2026-06-03 15:49:00.119079008 +0000 UTC m=+487.504146211" lastFinishedPulling="2026-06-03 15:49:02.240700264 +0000 UTC m=+489.625767467" observedRunningTime="2026-06-03 15:49:02.579091147 +0000 UTC m=+489.964158367" watchObservedRunningTime="2026-06-03 15:49:02.579970202 +0000 UTC m=+489.965037420" Jun 03 15:49:10.544897 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:10.544864 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zn9ll" Jun 03 15:49:13.556982 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.556950 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" Jun 03 15:49:13.645556 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.645523 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b"] Jun 03 15:49:13.648871 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.648850 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" Jun 03 15:49:13.651586 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.651560 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 03 15:49:13.651718 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.651700 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 03 15:49:13.651795 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.651742 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mpgcn\"" Jun 03 15:49:13.658371 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.658348 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b"] Jun 03 15:49:13.713530 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.713489 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88x2z\" (UniqueName: \"kubernetes.io/projected/d9f1aec7-bb5d-49af-ae13-e46213e44d05-kube-api-access-88x2z\") pod \"kuadrant-console-plugin-6cb54b5c86-dr22b\" (UID: \"d9f1aec7-bb5d-49af-ae13-e46213e44d05\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" Jun 03 15:49:13.713702 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.713565 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d9f1aec7-bb5d-49af-ae13-e46213e44d05-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-dr22b\" (UID: \"d9f1aec7-bb5d-49af-ae13-e46213e44d05\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" Jun 03 15:49:13.713702 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.713601 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9f1aec7-bb5d-49af-ae13-e46213e44d05-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-dr22b\" (UID: \"d9f1aec7-bb5d-49af-ae13-e46213e44d05\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" Jun 03 15:49:13.814584 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.814490 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d9f1aec7-bb5d-49af-ae13-e46213e44d05-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-dr22b\" (UID: \"d9f1aec7-bb5d-49af-ae13-e46213e44d05\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" Jun 03 15:49:13.814584 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.814541 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9f1aec7-bb5d-49af-ae13-e46213e44d05-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-dr22b\" (UID: \"d9f1aec7-bb5d-49af-ae13-e46213e44d05\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" Jun 03 15:49:13.814777 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.814597 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-88x2z\" (UniqueName: \"kubernetes.io/projected/d9f1aec7-bb5d-49af-ae13-e46213e44d05-kube-api-access-88x2z\") pod \"kuadrant-console-plugin-6cb54b5c86-dr22b\" (UID: \"d9f1aec7-bb5d-49af-ae13-e46213e44d05\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" Jun 03 15:49:13.814777 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:49:13.814685 2578 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jun 03 15:49:13.814777 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:49:13.814759 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f1aec7-bb5d-49af-ae13-e46213e44d05-plugin-serving-cert podName:d9f1aec7-bb5d-49af-ae13-e46213e44d05 nodeName:}" failed. No retries permitted until 2026-06-03 15:49:14.314742986 +0000 UTC m=+501.699810183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/d9f1aec7-bb5d-49af-ae13-e46213e44d05-plugin-serving-cert") pod "kuadrant-console-plugin-6cb54b5c86-dr22b" (UID: "d9f1aec7-bb5d-49af-ae13-e46213e44d05") : secret "plugin-serving-cert" not found Jun 03 15:49:13.815127 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.815108 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d9f1aec7-bb5d-49af-ae13-e46213e44d05-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-dr22b\" (UID: \"d9f1aec7-bb5d-49af-ae13-e46213e44d05\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" Jun 03 15:49:13.826095 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:13.826064 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88x2z\" (UniqueName: \"kubernetes.io/projected/d9f1aec7-bb5d-49af-ae13-e46213e44d05-kube-api-access-88x2z\") pod \"kuadrant-console-plugin-6cb54b5c86-dr22b\" (UID: \"d9f1aec7-bb5d-49af-ae13-e46213e44d05\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" Jun 03 15:49:14.318648 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:14.318607 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9f1aec7-bb5d-49af-ae13-e46213e44d05-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-dr22b\" (UID: \"d9f1aec7-bb5d-49af-ae13-e46213e44d05\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" Jun 03 15:49:14.321015 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:14.320994 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9f1aec7-bb5d-49af-ae13-e46213e44d05-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-dr22b\" (UID: \"d9f1aec7-bb5d-49af-ae13-e46213e44d05\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" Jun 03 15:49:14.560006 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:14.559944 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" Jun 03 15:49:14.691126 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:14.691095 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b"] Jun 03 15:49:14.693994 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:49:14.693960 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9f1aec7_bb5d_49af_ae13_e46213e44d05.slice/crio-b797332450b750460492397b76a2cad8838b8bf4bfe942b0bad6ddd76034367e WatchSource:0}: Error finding container b797332450b750460492397b76a2cad8838b8bf4bfe942b0bad6ddd76034367e: Status 404 returned error can't find the container with id b797332450b750460492397b76a2cad8838b8bf4bfe942b0bad6ddd76034367e Jun 03 15:49:15.594873 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:15.594834 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" event={"ID":"d9f1aec7-bb5d-49af-ae13-e46213e44d05","Type":"ContainerStarted","Data":"b797332450b750460492397b76a2cad8838b8bf4bfe942b0bad6ddd76034367e"} Jun 03 15:49:23.840822 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:23.840788 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4"] Jun 03 15:49:23.841298 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:23.841091 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" podUID="efe0b195-6563-4a69-9e36-f18c5edaaac0" containerName="manager" containerID="cri-o://576ae745b9fc23ebbdb3c9fbaa24c017a2148470ebeac4d0ea45b8fa96de57eb" gracePeriod=2 Jun 03 15:49:23.843660 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:23.843614 2578 status_manager.go:895] "Failed to get status for pod" podUID="efe0b195-6563-4a69-9e36-f18c5edaaac0" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" err="pods \"limitador-operator-controller-manager-85c4996f8c-8rsv4\" is forbidden: User \"system:node:ip-10-0-129-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-129.ec2.internal' and this object" Jun 03 15:49:23.843815 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:23.843762 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4"] Jun 03 15:49:23.870785 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:23.870754 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm"] Jun 03 15:49:23.871164 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:23.871140 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe0b195-6563-4a69-9e36-f18c5edaaac0" containerName="manager" Jun 03 15:49:23.871164 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:23.871159 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe0b195-6563-4a69-9e36-f18c5edaaac0" containerName="manager" Jun 03 15:49:23.871371 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:23.871260 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="efe0b195-6563-4a69-9e36-f18c5edaaac0" containerName="manager" Jun 03 15:49:23.874331 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:23.874308 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm" Jun 03 15:49:23.884595 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:23.884566 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm"] Jun 03 15:49:23.905117 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:23.905078 2578 status_manager.go:895] "Failed to get status for pod" podUID="efe0b195-6563-4a69-9e36-f18c5edaaac0" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" err="pods \"limitador-operator-controller-manager-85c4996f8c-8rsv4\" is forbidden: User \"system:node:ip-10-0-129-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-129.ec2.internal' and this object" Jun 03 15:49:24.003839 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.003806 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx25f\" (UniqueName: \"kubernetes.io/projected/34813cdd-aed6-4105-9571-64cab9cbdf56-kube-api-access-sx25f\") pod \"limitador-operator-controller-manager-85c4996f8c-sjznm\" (UID: \"34813cdd-aed6-4105-9571-64cab9cbdf56\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm" Jun 03 15:49:24.098990 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.098927 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" Jun 03 15:49:24.101557 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.101526 2578 status_manager.go:895] "Failed to get status for pod" podUID="efe0b195-6563-4a69-9e36-f18c5edaaac0" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" err="pods \"limitador-operator-controller-manager-85c4996f8c-8rsv4\" is forbidden: User \"system:node:ip-10-0-129-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-129.ec2.internal' and this object" Jun 03 15:49:24.104911 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.104881 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sx25f\" (UniqueName: \"kubernetes.io/projected/34813cdd-aed6-4105-9571-64cab9cbdf56-kube-api-access-sx25f\") pod \"limitador-operator-controller-manager-85c4996f8c-sjznm\" (UID: \"34813cdd-aed6-4105-9571-64cab9cbdf56\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm" Jun 03 15:49:24.127463 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.127434 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx25f\" (UniqueName: \"kubernetes.io/projected/34813cdd-aed6-4105-9571-64cab9cbdf56-kube-api-access-sx25f\") pod \"limitador-operator-controller-manager-85c4996f8c-sjznm\" (UID: \"34813cdd-aed6-4105-9571-64cab9cbdf56\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm" Jun 03 15:49:24.205632 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.205582 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vs7l2\" (UniqueName: \"kubernetes.io/projected/efe0b195-6563-4a69-9e36-f18c5edaaac0-kube-api-access-vs7l2\") pod \"efe0b195-6563-4a69-9e36-f18c5edaaac0\" (UID: \"efe0b195-6563-4a69-9e36-f18c5edaaac0\") " Jun 03 15:49:24.208028 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.207996 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe0b195-6563-4a69-9e36-f18c5edaaac0-kube-api-access-vs7l2" (OuterVolumeSpecName: "kube-api-access-vs7l2") pod "efe0b195-6563-4a69-9e36-f18c5edaaac0" (UID: "efe0b195-6563-4a69-9e36-f18c5edaaac0"). InnerVolumeSpecName "kube-api-access-vs7l2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 15:49:24.211216 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.211180 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm" Jun 03 15:49:24.306133 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.306109 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs7l2\" (UniqueName: \"kubernetes.io/projected/efe0b195-6563-4a69-9e36-f18c5edaaac0-kube-api-access-vs7l2\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:49:24.353434 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.353394 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm"] Jun 03 15:49:24.355496 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:49:24.355465 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34813cdd_aed6_4105_9571_64cab9cbdf56.slice/crio-4dc163633f4638572ccc8346ec3fdc019510d40b18688e2b157893b1db9d1ecc WatchSource:0}: Error finding container 4dc163633f4638572ccc8346ec3fdc019510d40b18688e2b157893b1db9d1ecc: Status 404 returned error can't find the container with id 4dc163633f4638572ccc8346ec3fdc019510d40b18688e2b157893b1db9d1ecc Jun 03 15:49:24.630919 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.630814 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm" event={"ID":"34813cdd-aed6-4105-9571-64cab9cbdf56","Type":"ContainerStarted","Data":"cf7dc69e49449f8c7a43afc6e522aa4895277afb0da8bcb8564090400b178be2"} Jun 03 15:49:24.630919 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.630864 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm" event={"ID":"34813cdd-aed6-4105-9571-64cab9cbdf56","Type":"ContainerStarted","Data":"4dc163633f4638572ccc8346ec3fdc019510d40b18688e2b157893b1db9d1ecc"} Jun 03 15:49:24.630919 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.630905 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm" Jun 03 15:49:24.632050 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.632020 2578 generic.go:358] "Generic (PLEG): container finished" podID="efe0b195-6563-4a69-9e36-f18c5edaaac0" containerID="576ae745b9fc23ebbdb3c9fbaa24c017a2148470ebeac4d0ea45b8fa96de57eb" exitCode=0 Jun 03 15:49:24.632182 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.632075 2578 scope.go:117] "RemoveContainer" containerID="576ae745b9fc23ebbdb3c9fbaa24c017a2148470ebeac4d0ea45b8fa96de57eb" Jun 03 15:49:24.632182 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.632073 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" Jun 03 15:49:24.678921 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.678423 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm" podStartSLOduration=1.6784034490000002 podStartE2EDuration="1.678403449s" podCreationTimestamp="2026-06-03 15:49:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 15:49:24.677836944 +0000 UTC m=+512.062904169" watchObservedRunningTime="2026-06-03 15:49:24.678403449 +0000 UTC m=+512.063470694" Jun 03 15:49:24.680556 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.680525 2578 status_manager.go:895] "Failed to get status for pod" podUID="efe0b195-6563-4a69-9e36-f18c5edaaac0" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" err="pods \"limitador-operator-controller-manager-85c4996f8c-8rsv4\" is forbidden: User \"system:node:ip-10-0-129-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-129.ec2.internal' and this object" Jun 03 15:49:24.682699 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:24.682663 2578 status_manager.go:895] "Failed to get status for pod" podUID="efe0b195-6563-4a69-9e36-f18c5edaaac0" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8rsv4" err="pods \"limitador-operator-controller-manager-85c4996f8c-8rsv4\" is forbidden: User \"system:node:ip-10-0-129-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-129.ec2.internal' and this object" Jun 03 15:49:25.112473 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:25.112439 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efe0b195-6563-4a69-9e36-f18c5edaaac0" path="/var/lib/kubelet/pods/efe0b195-6563-4a69-9e36-f18c5edaaac0/volumes" Jun 03 15:49:35.638214 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:35.638170 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-sjznm" Jun 03 15:49:38.483730 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:38.483501 2578 scope.go:117] "RemoveContainer" containerID="576ae745b9fc23ebbdb3c9fbaa24c017a2148470ebeac4d0ea45b8fa96de57eb" Jun 03 15:49:38.483984 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:49:38.483831 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"576ae745b9fc23ebbdb3c9fbaa24c017a2148470ebeac4d0ea45b8fa96de57eb\": container with ID starting with 576ae745b9fc23ebbdb3c9fbaa24c017a2148470ebeac4d0ea45b8fa96de57eb not found: ID does not exist" containerID="576ae745b9fc23ebbdb3c9fbaa24c017a2148470ebeac4d0ea45b8fa96de57eb" Jun 03 15:49:38.483984 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:38.483868 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576ae745b9fc23ebbdb3c9fbaa24c017a2148470ebeac4d0ea45b8fa96de57eb"} err="failed to get container status \"576ae745b9fc23ebbdb3c9fbaa24c017a2148470ebeac4d0ea45b8fa96de57eb\": rpc error: code = NotFound desc = could not find container \"576ae745b9fc23ebbdb3c9fbaa24c017a2148470ebeac4d0ea45b8fa96de57eb\": container with ID starting with 576ae745b9fc23ebbdb3c9fbaa24c017a2148470ebeac4d0ea45b8fa96de57eb not found: ID does not exist" Jun 03 15:49:38.684796 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:38.684764 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" event={"ID":"d9f1aec7-bb5d-49af-ae13-e46213e44d05","Type":"ContainerStarted","Data":"6fb6c2ab8b7d388248892e982c5dec1abbe5ef839e52644363ac3ac378fb40cc"} Jun 03 15:49:38.707480 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:38.707425 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-dr22b" podStartSLOduration=1.843491854 podStartE2EDuration="25.707410614s" podCreationTimestamp="2026-06-03 15:49:13 +0000 UTC" firstStartedPulling="2026-06-03 15:49:14.695340342 +0000 UTC m=+502.080407538" lastFinishedPulling="2026-06-03 15:49:38.559259097 +0000 UTC m=+525.944326298" observedRunningTime="2026-06-03 15:49:38.705483394 +0000 UTC m=+526.090550614" watchObservedRunningTime="2026-06-03 15:49:38.707410614 +0000 UTC m=+526.092477841" Jun 03 15:49:52.494814 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.494773 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf"] Jun 03 15:49:52.499322 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.499297 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.502328 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.502299 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9bx72\"" Jun 03 15:49:52.516040 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.516012 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf"] Jun 03 15:49:52.550153 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.550122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.550300 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.550155 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.550300 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.550175 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.550300 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.550195 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.550300 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.550274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.550435 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.550306 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.550435 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.550322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.550435 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.550344 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.550527 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.550433 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb8xc\" (UniqueName: \"kubernetes.io/projected/10bdeafe-e2d5-468c-ae61-9cba4ed72156-kube-api-access-fb8xc\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.651401 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.651368 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.651401 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.651403 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.651633 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.651421 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.651633 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.651439 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.651633 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.651493 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.651633 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.651524 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.651633 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.651553 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.651633 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.651589 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.651935 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.651647 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fb8xc\" (UniqueName: \"kubernetes.io/projected/10bdeafe-e2d5-468c-ae61-9cba4ed72156-kube-api-access-fb8xc\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.651935 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.651823 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.651935 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.651883 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.652093 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.651965 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.652165 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.652107 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.652401 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.652379 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.653845 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.653826 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.654073 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.654054 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.659481 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.659459 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10bdeafe-e2d5-468c-ae61-9cba4ed72156-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.659693 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.659673 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb8xc\" (UniqueName: \"kubernetes.io/projected/10bdeafe-e2d5-468c-ae61-9cba4ed72156-kube-api-access-fb8xc\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-767mf\" (UID: \"10bdeafe-e2d5-468c-ae61-9cba4ed72156\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.810991 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.810896 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:52.954367 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.951672 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf"] Jun 03 15:49:52.956227 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:49:52.956184 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bdeafe_e2d5_468c_ae61_9cba4ed72156.slice/crio-20adfc52b5d9cccaeb49bbf0d3af54e7f3060d938a6c8c85bc100d2ec128692b WatchSource:0}: Error finding container 20adfc52b5d9cccaeb49bbf0d3af54e7f3060d938a6c8c85bc100d2ec128692b: Status 404 returned error can't find the container with id 20adfc52b5d9cccaeb49bbf0d3af54e7f3060d938a6c8c85bc100d2ec128692b Jun 03 15:49:52.958290 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.958260 2578 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 03 15:49:52.958362 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.958322 2578 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 03 15:49:52.958362 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:52.958357 2578 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 03 15:49:53.731014 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:53.730974 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" event={"ID":"10bdeafe-e2d5-468c-ae61-9cba4ed72156","Type":"ContainerStarted","Data":"631a23a9daa336e6210e9a9e8a263eeb1f97c0fbc06e031c1e9a25d22a58fed8"} Jun 03 15:49:53.731014 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:53.731012 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" event={"ID":"10bdeafe-e2d5-468c-ae61-9cba4ed72156","Type":"ContainerStarted","Data":"20adfc52b5d9cccaeb49bbf0d3af54e7f3060d938a6c8c85bc100d2ec128692b"} Jun 03 15:49:53.762930 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:53.762869 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" podStartSLOduration=1.762850496 podStartE2EDuration="1.762850496s" podCreationTimestamp="2026-06-03 15:49:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 15:49:53.761362934 +0000 UTC m=+541.146430178" watchObservedRunningTime="2026-06-03 15:49:53.762850496 +0000 UTC m=+541.147917716" Jun 03 15:49:53.812000 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:53.811964 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:54.816406 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:54.816373 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:55.737505 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:55.737476 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:55.738422 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:55.738401 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-767mf" Jun 03 15:49:57.211567 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.211531 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-xcz7r"] Jun 03 15:49:57.356520 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.356482 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-xcz7r"] Jun 03 15:49:57.356520 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.356515 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-xcz7r"] Jun 03 15:49:57.356727 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.356616 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" Jun 03 15:49:57.359369 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.359342 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 03 15:49:57.391964 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.391930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqk5q\" (UniqueName: \"kubernetes.io/projected/115f07ee-166a-4e1f-9709-030c0c9b85cd-kube-api-access-wqk5q\") pod \"limitador-limitador-78c99df468-xcz7r\" (UID: \"115f07ee-166a-4e1f-9709-030c0c9b85cd\") " pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" Jun 03 15:49:57.392120 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.391987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/115f07ee-166a-4e1f-9709-030c0c9b85cd-config-file\") pod \"limitador-limitador-78c99df468-xcz7r\" (UID: \"115f07ee-166a-4e1f-9709-030c0c9b85cd\") " pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" Jun 03 15:49:57.493301 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.493262 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wqk5q\" (UniqueName: \"kubernetes.io/projected/115f07ee-166a-4e1f-9709-030c0c9b85cd-kube-api-access-wqk5q\") pod \"limitador-limitador-78c99df468-xcz7r\" (UID: \"115f07ee-166a-4e1f-9709-030c0c9b85cd\") " pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" Jun 03 15:49:57.493492 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.493321 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/115f07ee-166a-4e1f-9709-030c0c9b85cd-config-file\") pod \"limitador-limitador-78c99df468-xcz7r\" (UID: \"115f07ee-166a-4e1f-9709-030c0c9b85cd\") " pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" Jun 03 15:49:57.493860 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.493841 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/115f07ee-166a-4e1f-9709-030c0c9b85cd-config-file\") pod \"limitador-limitador-78c99df468-xcz7r\" (UID: \"115f07ee-166a-4e1f-9709-030c0c9b85cd\") " pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" Jun 03 15:49:57.502866 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.502841 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqk5q\" (UniqueName: \"kubernetes.io/projected/115f07ee-166a-4e1f-9709-030c0c9b85cd-kube-api-access-wqk5q\") pod \"limitador-limitador-78c99df468-xcz7r\" (UID: \"115f07ee-166a-4e1f-9709-030c0c9b85cd\") " pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" Jun 03 15:49:57.667098 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.667065 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" Jun 03 15:49:57.802395 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.802361 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-xcz7r"] Jun 03 15:49:57.805767 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:49:57.805737 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115f07ee_166a_4e1f_9709_030c0c9b85cd.slice/crio-c282c8af47619af07947f94a76015199a3823f1b656816728efd4cbe0eaf2445 WatchSource:0}: Error finding container c282c8af47619af07947f94a76015199a3823f1b656816728efd4cbe0eaf2445: Status 404 returned error can't find the container with id c282c8af47619af07947f94a76015199a3823f1b656816728efd4cbe0eaf2445 Jun 03 15:49:57.846685 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.846653 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-9xdw4"] Jun 03 15:49:57.884625 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.884594 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-9xdw4"] Jun 03 15:49:57.884774 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.884700 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-9xdw4" Jun 03 15:49:57.887390 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.887368 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hp85c\"" Jun 03 15:49:57.998428 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:57.998332 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjz5q\" (UniqueName: \"kubernetes.io/projected/751fd317-2d4b-419e-b08c-1257b0b3b115-kube-api-access-xjz5q\") pod \"authorino-7498df8756-9xdw4\" (UID: \"751fd317-2d4b-419e-b08c-1257b0b3b115\") " pod="kuadrant-system/authorino-7498df8756-9xdw4" Jun 03 15:49:58.099358 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:58.099320 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xjz5q\" (UniqueName: \"kubernetes.io/projected/751fd317-2d4b-419e-b08c-1257b0b3b115-kube-api-access-xjz5q\") pod \"authorino-7498df8756-9xdw4\" (UID: \"751fd317-2d4b-419e-b08c-1257b0b3b115\") " pod="kuadrant-system/authorino-7498df8756-9xdw4" Jun 03 15:49:58.108606 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:58.108570 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjz5q\" (UniqueName: \"kubernetes.io/projected/751fd317-2d4b-419e-b08c-1257b0b3b115-kube-api-access-xjz5q\") pod \"authorino-7498df8756-9xdw4\" (UID: \"751fd317-2d4b-419e-b08c-1257b0b3b115\") " pod="kuadrant-system/authorino-7498df8756-9xdw4" Jun 03 15:49:58.194649 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:58.194612 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-9xdw4" Jun 03 15:49:58.320999 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:58.320973 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-9xdw4"] Jun 03 15:49:58.323125 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:49:58.323099 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod751fd317_2d4b_419e_b08c_1257b0b3b115.slice/crio-087831552a60da21aa13f0a872cd06a38bb9c829c3d0ab7fa5d515e3608b5175 WatchSource:0}: Error finding container 087831552a60da21aa13f0a872cd06a38bb9c829c3d0ab7fa5d515e3608b5175: Status 404 returned error can't find the container with id 087831552a60da21aa13f0a872cd06a38bb9c829c3d0ab7fa5d515e3608b5175 Jun 03 15:49:58.750637 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:58.750596 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" event={"ID":"115f07ee-166a-4e1f-9709-030c0c9b85cd","Type":"ContainerStarted","Data":"c282c8af47619af07947f94a76015199a3823f1b656816728efd4cbe0eaf2445"} Jun 03 15:49:58.752048 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:49:58.752018 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-9xdw4" event={"ID":"751fd317-2d4b-419e-b08c-1257b0b3b115","Type":"ContainerStarted","Data":"087831552a60da21aa13f0a872cd06a38bb9c829c3d0ab7fa5d515e3608b5175"} Jun 03 15:50:03.771266 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:50:03.771226 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-9xdw4" event={"ID":"751fd317-2d4b-419e-b08c-1257b0b3b115","Type":"ContainerStarted","Data":"5180b2e830a8321d9f864b034b195f9286979875c24d5199d84bf570d7b7c2fe"} Jun 03 15:50:03.772644 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:50:03.772618 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" event={"ID":"115f07ee-166a-4e1f-9709-030c0c9b85cd","Type":"ContainerStarted","Data":"36efed88c5516297bf48cdbfb1298a8b9ed7d8c51ed5879c629a43e46bc42743"} Jun 03 15:50:03.772759 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:50:03.772738 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" Jun 03 15:50:03.791674 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:50:03.791620 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-9xdw4" podStartSLOduration=1.597213214 podStartE2EDuration="6.791605845s" podCreationTimestamp="2026-06-03 15:49:57 +0000 UTC" firstStartedPulling="2026-06-03 15:49:58.32448964 +0000 UTC m=+545.709556838" lastFinishedPulling="2026-06-03 15:50:03.518882268 +0000 UTC m=+550.903949469" observedRunningTime="2026-06-03 15:50:03.790361606 +0000 UTC m=+551.175428826" watchObservedRunningTime="2026-06-03 15:50:03.791605845 +0000 UTC m=+551.176673065" Jun 03 15:50:03.818899 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:50:03.818795 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" podStartSLOduration=1.182496302 podStartE2EDuration="6.818775756s" podCreationTimestamp="2026-06-03 15:49:57 +0000 UTC" firstStartedPulling="2026-06-03 15:49:57.807530566 +0000 UTC m=+545.192597766" lastFinishedPulling="2026-06-03 15:50:03.443810022 +0000 UTC m=+550.828877220" observedRunningTime="2026-06-03 15:50:03.817175017 +0000 UTC m=+551.202242237" watchObservedRunningTime="2026-06-03 15:50:03.818775756 +0000 UTC m=+551.203842976" Jun 03 15:50:14.779510 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:50:14.779473 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-xcz7r" Jun 03 15:50:53.012043 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:50:53.012015 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 15:50:53.012596 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:50:53.012564 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 15:51:25.784066 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:25.784030 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-9xdw4"] Jun 03 15:51:25.784622 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:25.784269 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-9xdw4" podUID="751fd317-2d4b-419e-b08c-1257b0b3b115" containerName="authorino" containerID="cri-o://5180b2e830a8321d9f864b034b195f9286979875c24d5199d84bf570d7b7c2fe" gracePeriod=30 Jun 03 15:51:26.035862 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:26.035785 2578 generic.go:358] "Generic (PLEG): container finished" podID="751fd317-2d4b-419e-b08c-1257b0b3b115" containerID="5180b2e830a8321d9f864b034b195f9286979875c24d5199d84bf570d7b7c2fe" exitCode=0 Jun 03 15:51:26.035862 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:26.035851 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-9xdw4" event={"ID":"751fd317-2d4b-419e-b08c-1257b0b3b115","Type":"ContainerDied","Data":"5180b2e830a8321d9f864b034b195f9286979875c24d5199d84bf570d7b7c2fe"} Jun 03 15:51:26.036041 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:26.035883 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-9xdw4" event={"ID":"751fd317-2d4b-419e-b08c-1257b0b3b115","Type":"ContainerDied","Data":"087831552a60da21aa13f0a872cd06a38bb9c829c3d0ab7fa5d515e3608b5175"} Jun 03 15:51:26.036041 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:26.035897 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="087831552a60da21aa13f0a872cd06a38bb9c829c3d0ab7fa5d515e3608b5175" Jun 03 15:51:26.036539 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:26.036521 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-9xdw4" Jun 03 15:51:26.129648 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:26.129612 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjz5q\" (UniqueName: \"kubernetes.io/projected/751fd317-2d4b-419e-b08c-1257b0b3b115-kube-api-access-xjz5q\") pod \"751fd317-2d4b-419e-b08c-1257b0b3b115\" (UID: \"751fd317-2d4b-419e-b08c-1257b0b3b115\") " Jun 03 15:51:26.131739 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:26.131709 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/751fd317-2d4b-419e-b08c-1257b0b3b115-kube-api-access-xjz5q" (OuterVolumeSpecName: "kube-api-access-xjz5q") pod "751fd317-2d4b-419e-b08c-1257b0b3b115" (UID: "751fd317-2d4b-419e-b08c-1257b0b3b115"). InnerVolumeSpecName "kube-api-access-xjz5q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 15:51:26.231136 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:26.231099 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjz5q\" (UniqueName: \"kubernetes.io/projected/751fd317-2d4b-419e-b08c-1257b0b3b115-kube-api-access-xjz5q\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:51:27.038466 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:27.038437 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-9xdw4" Jun 03 15:51:27.059721 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:27.059693 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-9xdw4"] Jun 03 15:51:27.063941 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:27.063915 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-9xdw4"] Jun 03 15:51:27.114398 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:27.114364 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="751fd317-2d4b-419e-b08c-1257b0b3b115" path="/var/lib/kubelet/pods/751fd317-2d4b-419e-b08c-1257b0b3b115/volumes" Jun 03 15:51:29.054283 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.054253 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68777b976-gg2m9"] Jun 03 15:51:29.054760 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.054699 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="751fd317-2d4b-419e-b08c-1257b0b3b115" containerName="authorino" Jun 03 15:51:29.054760 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.054717 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="751fd317-2d4b-419e-b08c-1257b0b3b115" containerName="authorino" Jun 03 15:51:29.054887 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.054808 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="751fd317-2d4b-419e-b08c-1257b0b3b115" containerName="authorino" Jun 03 15:51:29.056870 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.056846 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68777b976-gg2m9" Jun 03 15:51:29.059530 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.059510 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zv95v\"" Jun 03 15:51:29.059641 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.059534 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 03 15:51:29.070966 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.070940 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68777b976-gg2m9"] Jun 03 15:51:29.152419 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.152374 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4zlp\" (UniqueName: \"kubernetes.io/projected/37d5f8c3-a676-42ac-abf7-1272957576db-kube-api-access-p4zlp\") pod \"maas-controller-68777b976-gg2m9\" (UID: \"37d5f8c3-a676-42ac-abf7-1272957576db\") " pod="opendatahub/maas-controller-68777b976-gg2m9" Jun 03 15:51:29.253780 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.253746 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p4zlp\" (UniqueName: \"kubernetes.io/projected/37d5f8c3-a676-42ac-abf7-1272957576db-kube-api-access-p4zlp\") pod \"maas-controller-68777b976-gg2m9\" (UID: \"37d5f8c3-a676-42ac-abf7-1272957576db\") " pod="opendatahub/maas-controller-68777b976-gg2m9" Jun 03 15:51:29.263510 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.263479 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4zlp\" (UniqueName: \"kubernetes.io/projected/37d5f8c3-a676-42ac-abf7-1272957576db-kube-api-access-p4zlp\") pod \"maas-controller-68777b976-gg2m9\" (UID: \"37d5f8c3-a676-42ac-abf7-1272957576db\") " pod="opendatahub/maas-controller-68777b976-gg2m9" Jun 03 15:51:29.367360 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.367259 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68777b976-gg2m9" Jun 03 15:51:29.496158 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:29.496123 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68777b976-gg2m9"] Jun 03 15:51:29.498692 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:51:29.498654 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d5f8c3_a676_42ac_abf7_1272957576db.slice/crio-4af5404b866d12eb244905305c20a8d781ec9b08d4fa87ba09e060608dc82d3e WatchSource:0}: Error finding container 4af5404b866d12eb244905305c20a8d781ec9b08d4fa87ba09e060608dc82d3e: Status 404 returned error can't find the container with id 4af5404b866d12eb244905305c20a8d781ec9b08d4fa87ba09e060608dc82d3e Jun 03 15:51:30.049193 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:30.049154 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68777b976-gg2m9" event={"ID":"37d5f8c3-a676-42ac-abf7-1272957576db","Type":"ContainerStarted","Data":"4af5404b866d12eb244905305c20a8d781ec9b08d4fa87ba09e060608dc82d3e"} Jun 03 15:51:32.058962 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:32.058919 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68777b976-gg2m9" event={"ID":"37d5f8c3-a676-42ac-abf7-1272957576db","Type":"ContainerStarted","Data":"eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca"} Jun 03 15:51:32.059428 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:32.059001 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68777b976-gg2m9" Jun 03 15:51:32.088000 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:32.087920 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-68777b976-gg2m9" podStartSLOduration=0.68162033 podStartE2EDuration="3.087903423s" podCreationTimestamp="2026-06-03 15:51:29 +0000 UTC" firstStartedPulling="2026-06-03 15:51:29.499837113 +0000 UTC m=+636.884904310" lastFinishedPulling="2026-06-03 15:51:31.906120203 +0000 UTC m=+639.291187403" observedRunningTime="2026-06-03 15:51:32.087673451 +0000 UTC m=+639.472740671" watchObservedRunningTime="2026-06-03 15:51:32.087903423 +0000 UTC m=+639.472970641" Jun 03 15:51:43.069220 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:43.069171 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68777b976-gg2m9" Jun 03 15:51:43.680505 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:43.680466 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68777b976-gg2m9"] Jun 03 15:51:43.680729 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:43.680703 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-68777b976-gg2m9" podUID="37d5f8c3-a676-42ac-abf7-1272957576db" containerName="manager" containerID="cri-o://eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca" gracePeriod=30 Jun 03 15:51:43.918062 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:43.918039 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68777b976-gg2m9" Jun 03 15:51:44.079395 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:44.079343 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4zlp\" (UniqueName: \"kubernetes.io/projected/37d5f8c3-a676-42ac-abf7-1272957576db-kube-api-access-p4zlp\") pod \"37d5f8c3-a676-42ac-abf7-1272957576db\" (UID: \"37d5f8c3-a676-42ac-abf7-1272957576db\") " Jun 03 15:51:44.081469 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:44.081445 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37d5f8c3-a676-42ac-abf7-1272957576db-kube-api-access-p4zlp" (OuterVolumeSpecName: "kube-api-access-p4zlp") pod "37d5f8c3-a676-42ac-abf7-1272957576db" (UID: "37d5f8c3-a676-42ac-abf7-1272957576db"). InnerVolumeSpecName "kube-api-access-p4zlp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 15:51:44.098450 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:44.098417 2578 generic.go:358] "Generic (PLEG): container finished" podID="37d5f8c3-a676-42ac-abf7-1272957576db" containerID="eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca" exitCode=0 Jun 03 15:51:44.098549 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:44.098467 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68777b976-gg2m9" event={"ID":"37d5f8c3-a676-42ac-abf7-1272957576db","Type":"ContainerDied","Data":"eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca"} Jun 03 15:51:44.098549 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:44.098477 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68777b976-gg2m9" Jun 03 15:51:44.098549 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:44.098499 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68777b976-gg2m9" event={"ID":"37d5f8c3-a676-42ac-abf7-1272957576db","Type":"ContainerDied","Data":"4af5404b866d12eb244905305c20a8d781ec9b08d4fa87ba09e060608dc82d3e"} Jun 03 15:51:44.098549 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:44.098516 2578 scope.go:117] "RemoveContainer" containerID="eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca" Jun 03 15:51:44.107437 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:44.107419 2578 scope.go:117] "RemoveContainer" containerID="eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca" Jun 03 15:51:44.107691 ip-10-0-129-129 kubenswrapper[2578]: E0603 15:51:44.107673 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca\": container with ID starting with eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca not found: ID does not exist" containerID="eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca" Jun 03 15:51:44.107736 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:44.107701 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca"} err="failed to get container status \"eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca\": rpc error: code = NotFound desc = could not find container \"eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca\": container with ID starting with eee1e32e517d349c56d3409460c4995f16b174841323d50ed6040445cfdd7aca not found: ID does not exist" Jun 03 15:51:44.120495 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:44.120467 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68777b976-gg2m9"] Jun 03 15:51:44.124566 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:44.124547 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68777b976-gg2m9"] Jun 03 15:51:44.180497 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:44.180462 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4zlp\" (UniqueName: \"kubernetes.io/projected/37d5f8c3-a676-42ac-abf7-1272957576db-kube-api-access-p4zlp\") on node \"ip-10-0-129-129.ec2.internal\" DevicePath \"\"" Jun 03 15:51:45.111537 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:51:45.111504 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37d5f8c3-a676-42ac-abf7-1272957576db" path="/var/lib/kubelet/pods/37d5f8c3-a676-42ac-abf7-1272957576db/volumes" Jun 03 15:52:05.241509 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.241475 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-b6c896577-5plnj"] Jun 03 15:52:05.242081 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.241971 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37d5f8c3-a676-42ac-abf7-1272957576db" containerName="manager" Jun 03 15:52:05.242081 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.241992 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="37d5f8c3-a676-42ac-abf7-1272957576db" containerName="manager" Jun 03 15:52:05.242224 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.242083 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="37d5f8c3-a676-42ac-abf7-1272957576db" containerName="manager" Jun 03 15:52:05.245053 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.245031 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-b6c896577-5plnj" Jun 03 15:52:05.247432 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.247408 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 03 15:52:05.248162 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.248148 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-6fsqp\"" Jun 03 15:52:05.253445 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.253422 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-b6c896577-5plnj"] Jun 03 15:52:05.338669 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.338633 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42xk5\" (UniqueName: \"kubernetes.io/projected/8c8d7918-0f3f-499b-900a-03f03765e336-kube-api-access-42xk5\") pod \"maas-api-b6c896577-5plnj\" (UID: \"8c8d7918-0f3f-499b-900a-03f03765e336\") " pod="opendatahub/maas-api-b6c896577-5plnj" Jun 03 15:52:05.338842 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.338684 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c8d7918-0f3f-499b-900a-03f03765e336-maas-api-tls\") pod \"maas-api-b6c896577-5plnj\" (UID: \"8c8d7918-0f3f-499b-900a-03f03765e336\") " pod="opendatahub/maas-api-b6c896577-5plnj" Jun 03 15:52:05.440045 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.440008 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-42xk5\" (UniqueName: \"kubernetes.io/projected/8c8d7918-0f3f-499b-900a-03f03765e336-kube-api-access-42xk5\") pod \"maas-api-b6c896577-5plnj\" (UID: \"8c8d7918-0f3f-499b-900a-03f03765e336\") " pod="opendatahub/maas-api-b6c896577-5plnj" Jun 03 15:52:05.440252 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.440058 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c8d7918-0f3f-499b-900a-03f03765e336-maas-api-tls\") pod \"maas-api-b6c896577-5plnj\" (UID: \"8c8d7918-0f3f-499b-900a-03f03765e336\") " pod="opendatahub/maas-api-b6c896577-5plnj" Jun 03 15:52:05.442740 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.442689 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c8d7918-0f3f-499b-900a-03f03765e336-maas-api-tls\") pod \"maas-api-b6c896577-5plnj\" (UID: \"8c8d7918-0f3f-499b-900a-03f03765e336\") " pod="opendatahub/maas-api-b6c896577-5plnj" Jun 03 15:52:05.453842 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.453808 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42xk5\" (UniqueName: \"kubernetes.io/projected/8c8d7918-0f3f-499b-900a-03f03765e336-kube-api-access-42xk5\") pod \"maas-api-b6c896577-5plnj\" (UID: \"8c8d7918-0f3f-499b-900a-03f03765e336\") " pod="opendatahub/maas-api-b6c896577-5plnj" Jun 03 15:52:05.556134 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.556036 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-b6c896577-5plnj" Jun 03 15:52:05.703993 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.703958 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-b6c896577-5plnj"] Jun 03 15:52:05.704924 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:52:05.704899 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8d7918_0f3f_499b_900a_03f03765e336.slice/crio-0027f03c2034d1fd2fd5f50c6b9b521afc9c859f81c99586b215cc369ad5f3b3 WatchSource:0}: Error finding container 0027f03c2034d1fd2fd5f50c6b9b521afc9c859f81c99586b215cc369ad5f3b3: Status 404 returned error can't find the container with id 0027f03c2034d1fd2fd5f50c6b9b521afc9c859f81c99586b215cc369ad5f3b3 Jun 03 15:52:05.706059 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.706040 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 15:52:05.775256 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.775218 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-h7mwj"] Jun 03 15:52:05.779705 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.779686 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-h7mwj" Jun 03 15:52:05.784521 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.784497 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 03 15:52:05.784625 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.784554 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cpksq\"" Jun 03 15:52:05.794991 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.794965 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-h7mwj"] Jun 03 15:52:05.843961 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.843863 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk957\" (UniqueName: \"kubernetes.io/projected/b045baab-005e-4f74-a9a1-985cead6d9c3-kube-api-access-zk957\") pod \"payload-processing-58b6c8fdc7-h7mwj\" (UID: \"b045baab-005e-4f74-a9a1-985cead6d9c3\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-h7mwj" Jun 03 15:52:05.943689 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.943653 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-xcz7r"] Jun 03 15:52:05.944303 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.944278 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zk957\" (UniqueName: \"kubernetes.io/projected/b045baab-005e-4f74-a9a1-985cead6d9c3-kube-api-access-zk957\") pod \"payload-processing-58b6c8fdc7-h7mwj\" (UID: \"b045baab-005e-4f74-a9a1-985cead6d9c3\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-h7mwj" Jun 03 15:52:05.958319 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:05.958299 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk957\" (UniqueName: \"kubernetes.io/projected/b045baab-005e-4f74-a9a1-985cead6d9c3-kube-api-access-zk957\") pod \"payload-processing-58b6c8fdc7-h7mwj\" (UID: \"b045baab-005e-4f74-a9a1-985cead6d9c3\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-h7mwj" Jun 03 15:52:06.090651 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:06.090600 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-h7mwj" Jun 03 15:52:06.177522 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:06.177479 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-b6c896577-5plnj" event={"ID":"8c8d7918-0f3f-499b-900a-03f03765e336","Type":"ContainerStarted","Data":"0027f03c2034d1fd2fd5f50c6b9b521afc9c859f81c99586b215cc369ad5f3b3"} Jun 03 15:52:06.256842 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:52:06.256807 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb045baab_005e_4f74_a9a1_985cead6d9c3.slice/crio-a384de40173c1ecd241c2954dde662ddd4de5e40d4e70019f22073e7fcf29766 WatchSource:0}: Error finding container a384de40173c1ecd241c2954dde662ddd4de5e40d4e70019f22073e7fcf29766: Status 404 returned error can't find the container with id a384de40173c1ecd241c2954dde662ddd4de5e40d4e70019f22073e7fcf29766 Jun 03 15:52:06.257326 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:06.257302 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-h7mwj"] Jun 03 15:52:07.183459 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:07.183396 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-h7mwj" event={"ID":"b045baab-005e-4f74-a9a1-985cead6d9c3","Type":"ContainerStarted","Data":"a384de40173c1ecd241c2954dde662ddd4de5e40d4e70019f22073e7fcf29766"} Jun 03 15:52:08.188615 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:08.188574 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-b6c896577-5plnj" event={"ID":"8c8d7918-0f3f-499b-900a-03f03765e336","Type":"ContainerStarted","Data":"ddeb49096dc00643ab3169bb9a8a4724ea4adc7a8234853475a0b448cefee7fe"} Jun 03 15:52:08.189056 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:08.188698 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-b6c896577-5plnj" Jun 03 15:52:08.221740 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:08.221683 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-b6c896577-5plnj" podStartSLOduration=1.355927289 podStartE2EDuration="3.221664461s" podCreationTimestamp="2026-06-03 15:52:05 +0000 UTC" firstStartedPulling="2026-06-03 15:52:05.706158715 +0000 UTC m=+673.091225912" lastFinishedPulling="2026-06-03 15:52:07.571895887 +0000 UTC m=+674.956963084" observedRunningTime="2026-06-03 15:52:08.220277406 +0000 UTC m=+675.605344627" watchObservedRunningTime="2026-06-03 15:52:08.221664461 +0000 UTC m=+675.606731682" Jun 03 15:52:09.193576 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:09.193533 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-h7mwj" event={"ID":"b045baab-005e-4f74-a9a1-985cead6d9c3","Type":"ContainerStarted","Data":"9a540986eb40bae03beb41c05399e14bb1bcc7934812a945eee22e93bbc2d764"} Jun 03 15:52:09.229348 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:09.229290 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-h7mwj" podStartSLOduration=1.488055084 podStartE2EDuration="4.229275854s" podCreationTimestamp="2026-06-03 15:52:05 +0000 UTC" firstStartedPulling="2026-06-03 15:52:06.258767663 +0000 UTC m=+673.643834874" lastFinishedPulling="2026-06-03 15:52:08.999988444 +0000 UTC m=+676.385055644" observedRunningTime="2026-06-03 15:52:09.226762148 +0000 UTC m=+676.611829377" watchObservedRunningTime="2026-06-03 15:52:09.229275854 +0000 UTC m=+676.614343073" Jun 03 15:52:10.197126 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:10.197097 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-h7mwj" Jun 03 15:52:14.199160 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:14.199131 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-b6c896577-5plnj" Jun 03 15:52:21.202377 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:21.202345 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-h7mwj" Jun 03 15:52:54.042086 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.042047 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5bbc85db54-gvpjn"] Jun 03 15:52:54.050035 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.050013 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bbc85db54-gvpjn" Jun 03 15:52:54.052928 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.052870 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hp85c\"" Jun 03 15:52:54.052928 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.052870 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 03 15:52:54.052928 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.052904 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 03 15:52:54.054289 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.054266 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bbc85db54-gvpjn"] Jun 03 15:52:54.164044 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.164011 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8d6cdf67-66a8-4d08-b832-83aa277a4b9d-tls-cert\") pod \"authorino-5bbc85db54-gvpjn\" (UID: \"8d6cdf67-66a8-4d08-b832-83aa277a4b9d\") " pod="kuadrant-system/authorino-5bbc85db54-gvpjn" Jun 03 15:52:54.164254 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.164056 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dprjb\" (UniqueName: \"kubernetes.io/projected/8d6cdf67-66a8-4d08-b832-83aa277a4b9d-kube-api-access-dprjb\") pod \"authorino-5bbc85db54-gvpjn\" (UID: \"8d6cdf67-66a8-4d08-b832-83aa277a4b9d\") " pod="kuadrant-system/authorino-5bbc85db54-gvpjn" Jun 03 15:52:54.164254 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.164166 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/8d6cdf67-66a8-4d08-b832-83aa277a4b9d-oidc-ca\") pod \"authorino-5bbc85db54-gvpjn\" (UID: \"8d6cdf67-66a8-4d08-b832-83aa277a4b9d\") " pod="kuadrant-system/authorino-5bbc85db54-gvpjn" Jun 03 15:52:54.265333 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.265288 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8d6cdf67-66a8-4d08-b832-83aa277a4b9d-tls-cert\") pod \"authorino-5bbc85db54-gvpjn\" (UID: \"8d6cdf67-66a8-4d08-b832-83aa277a4b9d\") " pod="kuadrant-system/authorino-5bbc85db54-gvpjn" Jun 03 15:52:54.265333 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.265338 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dprjb\" (UniqueName: \"kubernetes.io/projected/8d6cdf67-66a8-4d08-b832-83aa277a4b9d-kube-api-access-dprjb\") pod \"authorino-5bbc85db54-gvpjn\" (UID: \"8d6cdf67-66a8-4d08-b832-83aa277a4b9d\") " pod="kuadrant-system/authorino-5bbc85db54-gvpjn" Jun 03 15:52:54.265597 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.265473 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/8d6cdf67-66a8-4d08-b832-83aa277a4b9d-oidc-ca\") pod \"authorino-5bbc85db54-gvpjn\" (UID: \"8d6cdf67-66a8-4d08-b832-83aa277a4b9d\") " pod="kuadrant-system/authorino-5bbc85db54-gvpjn" Jun 03 15:52:54.266048 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.266026 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/8d6cdf67-66a8-4d08-b832-83aa277a4b9d-oidc-ca\") pod \"authorino-5bbc85db54-gvpjn\" (UID: \"8d6cdf67-66a8-4d08-b832-83aa277a4b9d\") " pod="kuadrant-system/authorino-5bbc85db54-gvpjn" Jun 03 15:52:54.267844 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.267822 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8d6cdf67-66a8-4d08-b832-83aa277a4b9d-tls-cert\") pod \"authorino-5bbc85db54-gvpjn\" (UID: \"8d6cdf67-66a8-4d08-b832-83aa277a4b9d\") " pod="kuadrant-system/authorino-5bbc85db54-gvpjn" Jun 03 15:52:54.274104 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.274076 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dprjb\" (UniqueName: \"kubernetes.io/projected/8d6cdf67-66a8-4d08-b832-83aa277a4b9d-kube-api-access-dprjb\") pod \"authorino-5bbc85db54-gvpjn\" (UID: \"8d6cdf67-66a8-4d08-b832-83aa277a4b9d\") " pod="kuadrant-system/authorino-5bbc85db54-gvpjn" Jun 03 15:52:54.360569 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.360487 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bbc85db54-gvpjn" Jun 03 15:52:54.487232 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:54.487164 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bbc85db54-gvpjn"] Jun 03 15:52:54.490557 ip-10-0-129-129 kubenswrapper[2578]: W0603 15:52:54.490532 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d6cdf67_66a8_4d08_b832_83aa277a4b9d.slice/crio-a4c36d023e8cc655a27f98ba48a635a07228b92b875046218b991ea225575343 WatchSource:0}: Error finding container a4c36d023e8cc655a27f98ba48a635a07228b92b875046218b991ea225575343: Status 404 returned error can't find the container with id a4c36d023e8cc655a27f98ba48a635a07228b92b875046218b991ea225575343 Jun 03 15:52:55.347984 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:55.347943 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bbc85db54-gvpjn" event={"ID":"8d6cdf67-66a8-4d08-b832-83aa277a4b9d","Type":"ContainerStarted","Data":"f37531b6aa31d2bc8c91f60515e63d2e9c09eb9848e04ff429e6d146a0b94ff0"} Jun 03 15:52:55.347984 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:55.347986 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bbc85db54-gvpjn" event={"ID":"8d6cdf67-66a8-4d08-b832-83aa277a4b9d","Type":"ContainerStarted","Data":"a4c36d023e8cc655a27f98ba48a635a07228b92b875046218b991ea225575343"} Jun 03 15:52:55.370610 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:52:55.370549 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-5bbc85db54-gvpjn" podStartSLOduration=0.911030609 podStartE2EDuration="1.37053252s" podCreationTimestamp="2026-06-03 15:52:54 +0000 UTC" firstStartedPulling="2026-06-03 15:52:54.492324338 +0000 UTC m=+721.877391535" lastFinishedPulling="2026-06-03 15:52:54.951826244 +0000 UTC m=+722.336893446" observedRunningTime="2026-06-03 15:52:55.36804464 +0000 UTC m=+722.753111859" watchObservedRunningTime="2026-06-03 15:52:55.37053252 +0000 UTC m=+722.755599740" Jun 03 15:53:04.154993 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:53:04.154955 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-xcz7r"] Jun 03 15:53:14.248454 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:53:14.248422 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-xcz7r"] Jun 03 15:53:35.348292 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:53:35.348258 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-xcz7r"] Jun 03 15:53:53.445725 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:53:53.445683 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-xcz7r"] Jun 03 15:54:03.424698 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:54:03.424613 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-xcz7r"] Jun 03 15:54:17.358491 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:54:17.358450 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-xcz7r"] Jun 03 15:55:53.035369 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:55:53.035337 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 15:55:53.036842 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:55:53.036822 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 15:56:53.057241 ip-10-0-129-129 kubenswrapper[2578]: I0603 15:56:53.057190 2578 scope.go:117] "RemoveContainer" containerID="5180b2e830a8321d9f864b034b195f9286979875c24d5199d84bf570d7b7c2fe" Jun 03 16:00:16.028680 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:16.028636 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5bbc85db54-gvpjn_8d6cdf67-66a8-4d08-b832-83aa277a4b9d/authorino/0.log" Jun 03 16:00:20.769240 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:20.769183 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7888f9cd56-pgr29_bd517f90-303b-421d-9c50-7b079cd30f25/nginx/0.log" Jun 03 16:00:21.133684 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:21.133606 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-b6c896577-5plnj_8c8d7918-0f3f-499b-900a-03f03765e336/maas-api/0.log" Jun 03 16:00:21.494710 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:21.494683 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-bmvnr_fbcdca8b-8b39-46f0-ac72-64ac11529f65/server/0.log" Jun 03 16:00:21.620669 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:21.620642 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-jwgq2_57fd6338-2ffe-40bc-a9f4-5399b6bb441d/manager/0.log" Jun 03 16:00:21.856905 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:21.856831 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-548f4487f9-dnxvh_b23bd02b-221f-4d20-8887-430a22bf1f39/copy-manifests/0.log" Jun 03 16:00:21.874853 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:21.874814 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-548f4487f9-dnxvh_b23bd02b-221f-4d20-8887-430a22bf1f39/manager/0.log" Jun 03 16:00:23.513709 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:23.513683 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5bbc85db54-gvpjn_8d6cdf67-66a8-4d08-b832-83aa277a4b9d/authorino/0.log" Jun 03 16:00:23.738132 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:23.738102 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-zn9ll_1d5b300f-6c7e-4076-83e1-28ced13cf5b1/manager/0.log" Jun 03 16:00:23.853492 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:23.853411 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-dr22b_d9f1aec7-bb5d-49af-ae13-e46213e44d05/kuadrant-console-plugin/0.log" Jun 03 16:00:23.973115 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:23.973084 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-9vstf_5b94c742-a5b1-4870-8d94-bab39a7902cc/registry-server/0.log" Jun 03 16:00:24.217013 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:24.216976 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-xcz7r_115f07ee-166a-4e1f-9709-030c0c9b85cd/limitador/0.log" Jun 03 16:00:24.330221 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:24.330172 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-sjznm_34813cdd-aed6-4105-9571-64cab9cbdf56/manager/0.log" Jun 03 16:00:24.684595 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:24.684510 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bjx92k_7232360c-879b-4995-a402-c860535d44dc/istio-proxy/0.log" Jun 03 16:00:25.199506 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:25.199470 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-767mf_10bdeafe-e2d5-468c-ae61-9cba4ed72156/istio-proxy/0.log" Jun 03 16:00:25.317945 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:25.317913 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-h7mwj_b045baab-005e-4f74-a9a1-985cead6d9c3/payload-processing/0.log" Jun 03 16:00:30.776265 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:30.776228 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gjb2f/must-gather-xnrdt"] Jun 03 16:00:30.779501 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:30.779484 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gjb2f/must-gather-xnrdt" Jun 03 16:00:30.783276 ip-10-0-129-129 kubenswrapper[2578]: E0603 16:00:30.783240 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-9mlg5\" is forbidden: User \"system:node:ip-10-0-129-129.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-must-gather-gjb2f\": no relationship found between node 'ip-10-0-129-129.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-must-gather-gjb2f\"/\"default-dockercfg-9mlg5\"" type="*v1.Secret" Jun 03 16:00:30.783404 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:30.783293 2578 status_manager.go:895] "Failed to get status for pod" podUID="646566da-13a1-4aa2-9b0e-b950702bf2ff" pod="openshift-must-gather-gjb2f/must-gather-xnrdt" err="pods \"must-gather-xnrdt\" is forbidden: User \"system:node:ip-10-0-129-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gjb2f\": no relationship found between node 'ip-10-0-129-129.ec2.internal' and this object" Jun 03 16:00:30.783509 ip-10-0-129-129 kubenswrapper[2578]: E0603 16:00:30.783489 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-129.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-must-gather-gjb2f\": no relationship found between node 'ip-10-0-129-129.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-must-gather-gjb2f\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jun 03 16:00:30.783588 ip-10-0-129-129 kubenswrapper[2578]: E0603 16:00:30.783512 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-129.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-must-gather-gjb2f\": no relationship found between node 'ip-10-0-129-129.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-must-gather-gjb2f\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jun 03 16:00:30.815107 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:30.815070 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gjb2f/must-gather-xnrdt"] Jun 03 16:00:30.872694 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:30.872658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll75b\" (UniqueName: \"kubernetes.io/projected/646566da-13a1-4aa2-9b0e-b950702bf2ff-kube-api-access-ll75b\") pod \"must-gather-xnrdt\" (UID: \"646566da-13a1-4aa2-9b0e-b950702bf2ff\") " pod="openshift-must-gather-gjb2f/must-gather-xnrdt" Jun 03 16:00:30.872879 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:30.872704 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/646566da-13a1-4aa2-9b0e-b950702bf2ff-must-gather-output\") pod \"must-gather-xnrdt\" (UID: \"646566da-13a1-4aa2-9b0e-b950702bf2ff\") " pod="openshift-must-gather-gjb2f/must-gather-xnrdt" Jun 03 16:00:30.973495 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:30.973460 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ll75b\" (UniqueName: \"kubernetes.io/projected/646566da-13a1-4aa2-9b0e-b950702bf2ff-kube-api-access-ll75b\") pod \"must-gather-xnrdt\" (UID: \"646566da-13a1-4aa2-9b0e-b950702bf2ff\") " pod="openshift-must-gather-gjb2f/must-gather-xnrdt" Jun 03 16:00:30.973687 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:30.973519 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/646566da-13a1-4aa2-9b0e-b950702bf2ff-must-gather-output\") pod \"must-gather-xnrdt\" (UID: \"646566da-13a1-4aa2-9b0e-b950702bf2ff\") " pod="openshift-must-gather-gjb2f/must-gather-xnrdt" Jun 03 16:00:30.973868 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:30.973846 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/646566da-13a1-4aa2-9b0e-b950702bf2ff-must-gather-output\") pod \"must-gather-xnrdt\" (UID: \"646566da-13a1-4aa2-9b0e-b950702bf2ff\") " pod="openshift-must-gather-gjb2f/must-gather-xnrdt" Jun 03 16:00:31.980960 ip-10-0-129-129 kubenswrapper[2578]: E0603 16:00:31.980917 2578 projected.go:289] Couldn't get configMap openshift-must-gather-gjb2f/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jun 03 16:00:32.014090 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:32.014059 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gjb2f\"/\"openshift-service-ca.crt\"" Jun 03 16:00:32.021536 ip-10-0-129-129 kubenswrapper[2578]: E0603 16:00:32.021515 2578 projected.go:194] Error preparing data for projected volume kube-api-access-ll75b for pod openshift-must-gather-gjb2f/must-gather-xnrdt: failed to sync configmap cache: timed out waiting for the condition Jun 03 16:00:32.021615 ip-10-0-129-129 kubenswrapper[2578]: E0603 16:00:32.021588 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/646566da-13a1-4aa2-9b0e-b950702bf2ff-kube-api-access-ll75b podName:646566da-13a1-4aa2-9b0e-b950702bf2ff nodeName:}" failed. No retries permitted until 2026-06-03 16:00:32.521571437 +0000 UTC m=+1179.906638641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ll75b" (UniqueName: "kubernetes.io/projected/646566da-13a1-4aa2-9b0e-b950702bf2ff-kube-api-access-ll75b") pod "must-gather-xnrdt" (UID: "646566da-13a1-4aa2-9b0e-b950702bf2ff") : failed to sync configmap cache: timed out waiting for the condition Jun 03 16:00:32.254155 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:32.254076 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gjb2f\"/\"default-dockercfg-9mlg5\"" Jun 03 16:00:32.262474 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:32.262450 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gjb2f\"/\"kube-root-ca.crt\"" Jun 03 16:00:32.586274 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:32.586143 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ll75b\" (UniqueName: \"kubernetes.io/projected/646566da-13a1-4aa2-9b0e-b950702bf2ff-kube-api-access-ll75b\") pod \"must-gather-xnrdt\" (UID: \"646566da-13a1-4aa2-9b0e-b950702bf2ff\") " pod="openshift-must-gather-gjb2f/must-gather-xnrdt" Jun 03 16:00:32.588618 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:32.588585 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll75b\" (UniqueName: \"kubernetes.io/projected/646566da-13a1-4aa2-9b0e-b950702bf2ff-kube-api-access-ll75b\") pod \"must-gather-xnrdt\" (UID: \"646566da-13a1-4aa2-9b0e-b950702bf2ff\") " pod="openshift-must-gather-gjb2f/must-gather-xnrdt" Jun 03 16:00:32.888520 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:32.888422 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gjb2f/must-gather-xnrdt" Jun 03 16:00:33.021358 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:33.021331 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gjb2f/must-gather-xnrdt"] Jun 03 16:00:33.024229 ip-10-0-129-129 kubenswrapper[2578]: W0603 16:00:33.024174 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod646566da_13a1_4aa2_9b0e_b950702bf2ff.slice/crio-815de4461fb58942dccb1acfbd6c8a79efb79625266df0029443c88deefc1fff WatchSource:0}: Error finding container 815de4461fb58942dccb1acfbd6c8a79efb79625266df0029443c88deefc1fff: Status 404 returned error can't find the container with id 815de4461fb58942dccb1acfbd6c8a79efb79625266df0029443c88deefc1fff Jun 03 16:00:33.025872 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:33.025853 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:00:33.866764 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:33.866724 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjb2f/must-gather-xnrdt" event={"ID":"646566da-13a1-4aa2-9b0e-b950702bf2ff","Type":"ContainerStarted","Data":"815de4461fb58942dccb1acfbd6c8a79efb79625266df0029443c88deefc1fff"} Jun 03 16:00:34.871971 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:34.871930 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjb2f/must-gather-xnrdt" event={"ID":"646566da-13a1-4aa2-9b0e-b950702bf2ff","Type":"ContainerStarted","Data":"9ea7c42de3273b5c23879f393249a9f47fb76634b6c8d6e1a63bcb1d723fc3f5"} Jun 03 16:00:34.872474 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:34.871983 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjb2f/must-gather-xnrdt" event={"ID":"646566da-13a1-4aa2-9b0e-b950702bf2ff","Type":"ContainerStarted","Data":"80d4dd5e978ef29b576a5f46554c66e56f0401397d842107ef48028fd9f1b933"} Jun 03 16:00:34.890908 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:34.890839 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-gjb2f/must-gather-xnrdt" podStartSLOduration=4.116261044 podStartE2EDuration="4.890818216s" podCreationTimestamp="2026-06-03 16:00:30 +0000 UTC" firstStartedPulling="2026-06-03 16:00:33.026030803 +0000 UTC m=+1180.411097999" lastFinishedPulling="2026-06-03 16:00:33.800587959 +0000 UTC m=+1181.185655171" observedRunningTime="2026-06-03 16:00:34.889368229 +0000 UTC m=+1182.274435448" watchObservedRunningTime="2026-06-03 16:00:34.890818216 +0000 UTC m=+1182.275885436" Jun 03 16:00:35.644461 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:35.644428 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f27dk_576c5214-8806-4fd0-896c-0b055986dab4/global-pull-secret-syncer/0.log" Jun 03 16:00:35.744704 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:35.744671 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4dfqm_a239c904-33ce-4c96-88ce-c976799776e8/konnectivity-agent/0.log" Jun 03 16:00:35.860904 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:35.860852 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-129.ec2.internal_43488e4976b4b669cc3a2c15df77a88a/haproxy/0.log" Jun 03 16:00:40.177529 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:40.177492 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5bbc85db54-gvpjn_8d6cdf67-66a8-4d08-b832-83aa277a4b9d/authorino/0.log" Jun 03 16:00:40.313999 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:40.313957 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-zn9ll_1d5b300f-6c7e-4076-83e1-28ced13cf5b1/manager/0.log" Jun 03 16:00:40.366842 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:40.366811 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-dr22b_d9f1aec7-bb5d-49af-ae13-e46213e44d05/kuadrant-console-plugin/0.log" Jun 03 16:00:40.406960 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:40.406926 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-9vstf_5b94c742-a5b1-4870-8d94-bab39a7902cc/registry-server/0.log" Jun 03 16:00:40.491310 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:40.491278 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-xcz7r_115f07ee-166a-4e1f-9709-030c0c9b85cd/limitador/0.log" Jun 03 16:00:40.528653 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:40.528614 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-sjznm_34813cdd-aed6-4105-9571-64cab9cbdf56/manager/0.log" Jun 03 16:00:42.431051 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:42.431017 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5txw4_6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3/node-exporter/0.log" Jun 03 16:00:42.457497 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:42.457469 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5txw4_6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3/kube-rbac-proxy/0.log" Jun 03 16:00:42.486499 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:42.486470 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5txw4_6082d5ec-f4d3-4d2e-b4cd-aa32c27f1da3/init-textfile/0.log" Jun 03 16:00:44.732181 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.732133 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2"] Jun 03 16:00:44.739989 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.739949 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.742832 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.742800 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2"] Jun 03 16:00:44.808655 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.808620 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3b70001-929b-4d18-b8d4-ca447f10cc0b-sys\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.808655 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.808660 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c3b70001-929b-4d18-b8d4-ca447f10cc0b-proc\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.808878 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.808766 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c26sm\" (UniqueName: \"kubernetes.io/projected/c3b70001-929b-4d18-b8d4-ca447f10cc0b-kube-api-access-c26sm\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.808878 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.808801 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3b70001-929b-4d18-b8d4-ca447f10cc0b-lib-modules\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.808878 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.808860 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c3b70001-929b-4d18-b8d4-ca447f10cc0b-podres\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.910191 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.910151 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c26sm\" (UniqueName: \"kubernetes.io/projected/c3b70001-929b-4d18-b8d4-ca447f10cc0b-kube-api-access-c26sm\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.910416 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.910242 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3b70001-929b-4d18-b8d4-ca447f10cc0b-lib-modules\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.910416 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.910316 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c3b70001-929b-4d18-b8d4-ca447f10cc0b-podres\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.910416 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.910358 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3b70001-929b-4d18-b8d4-ca447f10cc0b-sys\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.910416 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.910402 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3b70001-929b-4d18-b8d4-ca447f10cc0b-lib-modules\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.910646 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.910408 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c3b70001-929b-4d18-b8d4-ca447f10cc0b-proc\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.910646 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.910481 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c3b70001-929b-4d18-b8d4-ca447f10cc0b-proc\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.910646 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.910529 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c3b70001-929b-4d18-b8d4-ca447f10cc0b-podres\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.910646 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.910545 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3b70001-929b-4d18-b8d4-ca447f10cc0b-sys\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:44.919498 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:44.919469 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c26sm\" (UniqueName: \"kubernetes.io/projected/c3b70001-929b-4d18-b8d4-ca447f10cc0b-kube-api-access-c26sm\") pod \"perf-node-gather-daemonset-5sbh2\" (UID: \"c3b70001-929b-4d18-b8d4-ca447f10cc0b\") " pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:45.055131 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:45.055037 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:45.215873 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:45.214268 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2"] Jun 03 16:00:45.219955 ip-10-0-129-129 kubenswrapper[2578]: W0603 16:00:45.219922 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc3b70001_929b_4d18_b8d4_ca447f10cc0b.slice/crio-f1a28f71924f7bb87379052fa7b6042299a6532aa14688ac0ffac67cac3965f5 WatchSource:0}: Error finding container f1a28f71924f7bb87379052fa7b6042299a6532aa14688ac0ffac67cac3965f5: Status 404 returned error can't find the container with id f1a28f71924f7bb87379052fa7b6042299a6532aa14688ac0ffac67cac3965f5 Jun 03 16:00:45.454053 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:45.454009 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-rjdvs_aafc4657-83d5-4539-96fb-abcf974a2180/download-server/0.log" Jun 03 16:00:45.919541 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:45.919504 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" event={"ID":"c3b70001-929b-4d18-b8d4-ca447f10cc0b","Type":"ContainerStarted","Data":"3c18af4d0601df8adb0d961a7e65bc4aabff11ac268a07153499f9f3f2d49e34"} Jun 03 16:00:45.920120 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:45.920084 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" event={"ID":"c3b70001-929b-4d18-b8d4-ca447f10cc0b","Type":"ContainerStarted","Data":"f1a28f71924f7bb87379052fa7b6042299a6532aa14688ac0ffac67cac3965f5"} Jun 03 16:00:45.920295 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:45.920281 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:46.951034 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:46.951003 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n4lvz_6cb3b438-55f9-4194-ba2a-e91afec5a1e4/dns/0.log" Jun 03 16:00:46.972703 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:46.972673 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n4lvz_6cb3b438-55f9-4194-ba2a-e91afec5a1e4/kube-rbac-proxy/0.log" Jun 03 16:00:46.998792 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:46.998765 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4hx4n_e3c78079-acba-48d7-a32a-2dd02cb04d98/dns-node-resolver/0.log" Jun 03 16:00:47.661603 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:47.661575 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-58fbcd44d-rw7xq_34c38475-cfdd-4e86-ae06-cf991fd4aa53/registry/0.log" Jun 03 16:00:47.684013 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:47.683975 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gh9z6_22b6b156-cdf5-4696-b64c-fec58fcc28e5/node-ca/0.log" Jun 03 16:00:48.627081 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:48.627045 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bjx92k_7232360c-879b-4995-a402-c860535d44dc/istio-proxy/0.log" Jun 03 16:00:48.835612 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:48.835578 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-767mf_10bdeafe-e2d5-468c-ae61-9cba4ed72156/istio-proxy/0.log" Jun 03 16:00:48.862303 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:48.862268 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-h7mwj_b045baab-005e-4f74-a9a1-985cead6d9c3/payload-processing/0.log" Jun 03 16:00:49.458739 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:49.458705 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-96wm8_5ca43b94-b17e-4492-a995-30971a21f97b/serve-healthcheck-canary/0.log" Jun 03 16:00:49.993259 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:49.993229 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k2lgj_d29a2368-495f-4637-8663-32d7687e81ac/kube-rbac-proxy/0.log" Jun 03 16:00:50.017501 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:50.017467 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k2lgj_d29a2368-495f-4637-8663-32d7687e81ac/exporter/0.log" Jun 03 16:00:50.045047 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:50.045016 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k2lgj_d29a2368-495f-4637-8663-32d7687e81ac/extractor/0.log" Jun 03 16:00:52.357002 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:52.356964 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7888f9cd56-pgr29_bd517f90-303b-421d-9c50-7b079cd30f25/nginx/0.log" Jun 03 16:00:52.454405 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:52.454372 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-b6c896577-5plnj_8c8d7918-0f3f-499b-900a-03f03765e336/maas-api/0.log" Jun 03 16:00:52.538148 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:52.538113 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-bmvnr_fbcdca8b-8b39-46f0-ac72-64ac11529f65/server/0.log" Jun 03 16:00:52.612968 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:52.612897 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-jwgq2_57fd6338-2ffe-40bc-a9f4-5399b6bb441d/manager/0.log" Jun 03 16:00:52.790577 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:52.790535 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-548f4487f9-dnxvh_b23bd02b-221f-4d20-8887-430a22bf1f39/manager/0.log" Jun 03 16:00:52.849055 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:52.849027 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-548f4487f9-dnxvh_b23bd02b-221f-4d20-8887-430a22bf1f39/copy-manifests/0.log" Jun 03 16:00:52.937158 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:52.937089 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" Jun 03 16:00:52.955938 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:52.955893 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-gjb2f/perf-node-gather-daemonset-5sbh2" podStartSLOduration=8.955879061 podStartE2EDuration="8.955879061s" podCreationTimestamp="2026-06-03 16:00:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:00:45.939039741 +0000 UTC m=+1193.324106960" watchObservedRunningTime="2026-06-03 16:00:52.955879061 +0000 UTC m=+1200.340946328" Jun 03 16:00:53.060536 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:53.060503 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 16:00:53.062340 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:53.062317 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kdw5d_13e88df3-970c-472f-9f44-1a2102fa3695/ovn-acl-logging/0.log" Jun 03 16:00:54.354509 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:00:54.354479 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-f96cfff67-7kvtm_606b6b69-c604-4a32-a449-7178a7b1a866/manager/0.log" Jun 03 16:01:01.231614 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:01:01.231588 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jmlt_a98bd1db-934b-442e-8851-e8a60695aa8d/kube-multus-additional-cni-plugins/0.log" Jun 03 16:01:01.255365 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:01:01.255338 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jmlt_a98bd1db-934b-442e-8851-e8a60695aa8d/egress-router-binary-copy/0.log" Jun 03 16:01:01.279035 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:01:01.279006 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jmlt_a98bd1db-934b-442e-8851-e8a60695aa8d/cni-plugins/0.log" Jun 03 16:01:01.303727 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:01:01.303699 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jmlt_a98bd1db-934b-442e-8851-e8a60695aa8d/bond-cni-plugin/0.log" Jun 03 16:01:01.325331 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:01:01.325305 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jmlt_a98bd1db-934b-442e-8851-e8a60695aa8d/routeoverride-cni/0.log" Jun 03 16:01:01.348928 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:01:01.348894 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jmlt_a98bd1db-934b-442e-8851-e8a60695aa8d/whereabouts-cni-bincopy/0.log" Jun 03 16:01:01.370970 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:01:01.370945 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4jmlt_a98bd1db-934b-442e-8851-e8a60695aa8d/whereabouts-cni/0.log" Jun 03 16:01:01.787488 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:01:01.787448 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-v6zjk_b47a65fc-0f6c-4a56-96eb-fd5065fa8a0e/kube-multus/0.log" Jun 03 16:01:01.913748 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:01:01.913714 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t7kx5_a9d8c3c0-82e0-48b5-909a-a7887d92baa9/network-metrics-daemon/0.log" Jun 03 16:01:01.940983 ip-10-0-129-129 kubenswrapper[2578]: I0603 16:01:01.940956 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t7kx5_a9d8c3c0-82e0-48b5-909a-a7887d92baa9/kube-rbac-proxy/0.log"