Jun 02 19:55:12.566091 ip-10-0-129-157 systemd[1]: Starting Kubernetes Kubelet... Jun 02 19:55:12.993274 ip-10-0-129-157 kubenswrapper[2548]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 19:55:12.993274 ip-10-0-129-157 kubenswrapper[2548]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 19:55:12.993274 ip-10-0-129-157 kubenswrapper[2548]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 19:55:12.993274 ip-10-0-129-157 kubenswrapper[2548]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 19:55:12.993274 ip-10-0-129-157 kubenswrapper[2548]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 19:55:12.993982 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.993888 2548 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 19:55:12.997314 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997295 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:55:12.997314 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997314 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:55:12.997314 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997318 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997321 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997325 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997328 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997337 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997341 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997344 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997348 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997352 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997355 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997358 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997360 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997363 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997366 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997368 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997371 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997374 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997377 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997379 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997382 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:55:12.997420 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997385 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997388 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997390 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997393 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997396 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997398 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997401 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997404 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997406 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997409 2548 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997411 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997414 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997416 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997419 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997422 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997427 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997429 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997432 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997435 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:55:12.997930 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997439 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997443 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997445 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997448 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997452 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997455 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997458 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997461 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997463 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997466 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997469 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997471 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997474 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997477 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997479 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997482 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997485 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997488 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997490 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997493 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:55:12.998406 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997495 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997498 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997501 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997505 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997507 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997510 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997512 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997516 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997519 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997522 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997524 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997527 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997530 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997532 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997534 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997537 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997539 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997542 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997544 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997547 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:55:12.998910 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997549 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997552 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997554 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997557 2548 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997559 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997974 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997980 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997982 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997985 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997988 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997991 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997994 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.997997 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998000 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998003 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998005 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998008 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998011 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998013 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998016 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:55:12.999401 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998018 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998022 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998024 2548 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998027 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998029 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998032 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998034 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998037 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998039 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998042 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998045 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998047 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998050 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998053 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998056 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998059 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998061 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998063 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998066 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998069 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:55:12.999897 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998071 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998074 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998077 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998079 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998082 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998085 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998087 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998090 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998092 2548 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998095 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998097 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998100 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998102 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998104 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998107 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998110 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998114 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998116 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998119 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998121 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:55:13.000398 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998123 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998126 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998128 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998132 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998134 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998137 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998139 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998142 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998145 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998148 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998150 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998153 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998157 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998160 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998162 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998165 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998167 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998170 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998173 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998176 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:55:13.000912 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998178 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998181 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998183 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998186 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998188 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998191 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998193 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998196 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998199 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998201 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.998205 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999107 2548 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999117 2548 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999123 2548 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999128 2548 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999133 2548 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999136 2548 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999140 2548 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999145 2548 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999152 2548 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 19:55:13.001434 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999155 2548 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999159 2548 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999162 2548 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999165 2548 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999168 2548 flags.go:64] FLAG: --cgroup-root="" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999171 2548 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999174 2548 flags.go:64] FLAG: --client-ca-file="" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999177 2548 flags.go:64] FLAG: --cloud-config="" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999180 2548 flags.go:64] FLAG: --cloud-provider="external" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999183 2548 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999189 2548 flags.go:64] FLAG: --cluster-domain="" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999192 2548 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999195 2548 flags.go:64] FLAG: --config-dir="" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999198 2548 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999201 2548 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999205 2548 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999208 2548 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999211 2548 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999215 2548 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999218 2548 flags.go:64] FLAG: --contention-profiling="false" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999221 2548 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999224 2548 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999227 2548 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999230 2548 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999234 2548 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 19:55:13.002045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999237 2548 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999240 2548 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999243 2548 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999247 2548 flags.go:64] FLAG: --enable-server="true" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999249 2548 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999254 2548 flags.go:64] FLAG: --event-burst="100" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999258 2548 flags.go:64] FLAG: --event-qps="50" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999261 2548 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999264 2548 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999267 2548 flags.go:64] FLAG: --eviction-hard="" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999271 2548 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999274 2548 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999277 2548 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999280 2548 flags.go:64] FLAG: --eviction-soft="" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999284 2548 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999286 2548 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999289 2548 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999292 2548 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999295 2548 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999298 2548 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999301 2548 flags.go:64] FLAG: --feature-gates="" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999305 2548 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999308 2548 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999311 2548 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999315 2548 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999318 2548 flags.go:64] FLAG: --healthz-port="10248" Jun 02 19:55:13.002645 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999321 2548 flags.go:64] FLAG: --help="false" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999324 2548 flags.go:64] FLAG: --hostname-override="ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999328 2548 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999332 2548 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999335 2548 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999338 2548 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999341 2548 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999344 2548 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999347 2548 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999349 2548 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999352 2548 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999356 2548 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999361 2548 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999364 2548 flags.go:64] FLAG: --kube-reserved="" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999367 2548 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999370 2548 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999373 2548 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999376 2548 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999379 2548 flags.go:64] FLAG: --lock-file="" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999382 2548 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999385 2548 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999388 2548 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999394 2548 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 19:55:13.003321 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999397 2548 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999400 2548 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999403 2548 flags.go:64] FLAG: --logging-format="text" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999407 2548 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999410 2548 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999413 2548 flags.go:64] FLAG: --manifest-url="" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999416 2548 flags.go:64] FLAG: --manifest-url-header="" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999421 2548 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999424 2548 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999430 2548 flags.go:64] FLAG: --max-pods="110" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999433 2548 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999436 2548 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999441 2548 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999444 2548 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999447 2548 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999450 2548 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999453 2548 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999461 2548 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999464 2548 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999467 2548 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999470 2548 flags.go:64] FLAG: --pod-cidr="" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999474 2548 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999480 2548 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999483 2548 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 19:55:13.003907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999486 2548 flags.go:64] FLAG: --pods-per-core="0" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999488 2548 flags.go:64] FLAG: --port="10250" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999491 2548 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999494 2548 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-087506f7d665530f4" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999497 2548 flags.go:64] FLAG: --qos-reserved="" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999500 2548 flags.go:64] FLAG: --read-only-port="10255" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999503 2548 flags.go:64] FLAG: --register-node="true" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999506 2548 flags.go:64] FLAG: --register-schedulable="true" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999509 2548 flags.go:64] FLAG: --register-with-taints="" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999513 2548 flags.go:64] FLAG: --registry-burst="10" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999515 2548 flags.go:64] FLAG: --registry-qps="5" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999518 2548 flags.go:64] FLAG: --reserved-cpus="" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999521 2548 flags.go:64] FLAG: --reserved-memory="" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999525 2548 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999528 2548 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999531 2548 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999533 2548 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999538 2548 flags.go:64] FLAG: --runonce="false" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999541 2548 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999544 2548 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999547 2548 flags.go:64] FLAG: --seccomp-default="false" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999550 2548 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999554 2548 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999557 2548 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999560 2548 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999563 2548 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 19:55:13.004514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999567 2548 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999569 2548 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999572 2548 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999575 2548 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999578 2548 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999581 2548 flags.go:64] FLAG: --system-cgroups="" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999584 2548 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999590 2548 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999592 2548 flags.go:64] FLAG: --tls-cert-file="" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999596 2548 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999601 2548 flags.go:64] FLAG: --tls-min-version="" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999603 2548 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999606 2548 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999609 2548 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999612 2548 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999615 2548 flags.go:64] FLAG: --v="2" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999619 2548 flags.go:64] FLAG: --version="false" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999623 2548 flags.go:64] FLAG: --vmodule="" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999628 2548 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:12.999632 2548 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999730 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999734 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999750 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999759 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:55:13.005161 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999762 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999765 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999768 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999771 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999774 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999777 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999779 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999782 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999784 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999787 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999790 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999792 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999795 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999797 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999800 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999803 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999805 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999808 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999811 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:55:13.005772 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999815 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999819 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999821 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999825 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999827 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999830 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999833 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999836 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999838 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999841 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999844 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999847 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999851 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999855 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999859 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999862 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999864 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999867 2548 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999870 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:55:13.006286 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999873 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999875 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999878 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999880 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999883 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999885 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999888 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999891 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999893 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999895 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999898 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999901 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999903 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999906 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999909 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999911 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999914 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999916 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999919 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999922 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:55:13.006832 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999925 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999927 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999930 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999933 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999935 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999938 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999941 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999943 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999946 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999948 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999952 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999955 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999958 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999960 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999963 2548 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999966 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999968 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999971 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999973 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999976 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999979 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:55:13.007310 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999982 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999984 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:12.999987 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.000538 2548 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.007610 2548 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.007631 2548 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007681 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007686 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007690 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007693 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007696 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007699 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007702 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007705 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007708 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:55:13.007844 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007710 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007713 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007716 2548 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007719 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007721 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007724 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007727 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007730 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007732 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007735 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007752 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007755 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007758 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007760 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007763 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007766 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007768 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007771 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007773 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007776 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:55:13.008223 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007779 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007782 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007787 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007789 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007792 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007795 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007797 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007800 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007803 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007805 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007808 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007811 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007813 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007816 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007818 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007821 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007823 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007826 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007829 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007832 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:55:13.008709 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007834 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007837 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007839 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007842 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007844 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007849 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007854 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007857 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007860 2548 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007863 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007866 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007869 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007872 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007875 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007878 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007882 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007884 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007887 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007889 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007892 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:55:13.009230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007894 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007897 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007899 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007902 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007905 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007907 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007910 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007912 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007915 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007917 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007920 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007922 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007925 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007927 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007930 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007933 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:55:13.009725 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.007937 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.007943 2548 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008045 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008050 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008053 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008056 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008060 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008064 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008067 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008070 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008073 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008076 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008079 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008082 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008085 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:55:13.010140 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008088 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008091 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008093 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008096 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008099 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008101 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008104 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008106 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008109 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008112 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008114 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008116 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008119 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008121 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008124 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008127 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008130 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008132 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008135 2548 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008137 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:55:13.010523 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008140 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008143 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008145 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008148 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008151 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008153 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008156 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008159 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008161 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008164 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008167 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008170 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008172 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008175 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008177 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008179 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008182 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008184 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008187 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008189 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:55:13.011033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008192 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008194 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008197 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008199 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008203 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008206 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008209 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008212 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008215 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008218 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008221 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008223 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008226 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008229 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008233 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008236 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008240 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008243 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008246 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008248 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:55:13.011543 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008251 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008253 2548 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008256 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008259 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008261 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008264 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008266 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008269 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008271 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008274 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008276 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008279 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:13.008281 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.008286 2548 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 19:55:13.012033 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.008919 2548 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 19:55:13.013220 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.013203 2548 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 19:55:13.014373 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.014361 2548 server.go:1019] "Starting client certificate rotation" Jun 02 19:55:13.014481 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.014461 2548 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 19:55:13.014520 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.014504 2548 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 19:55:13.037477 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.037452 2548 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 19:55:13.040732 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.040696 2548 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 19:55:13.053762 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.053708 2548 log.go:25] "Validated CRI v1 runtime API" Jun 02 19:55:13.061341 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.061316 2548 log.go:25] "Validated CRI v1 image API" Jun 02 19:55:13.064015 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.063981 2548 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 19:55:13.067986 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.067962 2548 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 928b2561-ff3e-4a54-b9af-8abc6bda4696:/dev/nvme0n1p3 fda2f854-d269-4949-83d2-252859e545dc:/dev/nvme0n1p4] Jun 02 19:55:13.068082 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.067983 2548 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 19:55:13.072924 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.072652 2548 manager.go:217] Machine: {Timestamp:2026-06-02 19:55:13.072140795 +0000 UTC m=+0.389354061 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099639 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21d1c87ad332e0a6a94b1b866039c7 SystemUUID:ec21d1c8-7ad3-32e0-a6a9-4b1b866039c7 BootID:958ac12b-3f37-476e-8702-5910f20a5a58 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0e:b2:6a:cf:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0e:b2:6a:cf:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:d7:33:b4:a1:87 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 19:55:13.073984 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.073968 2548 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 19:55:13.074095 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.074080 2548 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 19:55:13.075810 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.075778 2548 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 19:55:13.075969 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.075814 2548 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-157.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 19:55:13.076017 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.075976 2548 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 19:55:13.076017 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.075985 2548 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 19:55:13.076017 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.075993 2548 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 19:55:13.076720 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.076706 2548 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 19:55:13.078070 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.078057 2548 state_mem.go:36] "Initialized new in-memory state store" Jun 02 19:55:13.078194 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.078185 2548 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 19:55:13.080759 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.080730 2548 kubelet.go:491] "Attempting to sync node with API server" Jun 02 19:55:13.080801 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.080766 2548 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 19:55:13.080801 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.080788 2548 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 19:55:13.080863 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.080803 2548 kubelet.go:397] "Adding apiserver pod source" Jun 02 19:55:13.080863 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.080815 2548 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 19:55:13.081925 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.081911 2548 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 19:55:13.081974 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.081933 2548 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 19:55:13.085182 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.085155 2548 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 19:55:13.087142 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.087123 2548 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 19:55:13.087207 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.087192 2548 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 19:55:13.089458 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.089443 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 19:55:13.089509 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.089467 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 19:55:13.089509 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.089476 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 19:55:13.089509 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.089485 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 19:55:13.089509 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.089497 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 19:55:13.089509 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.089506 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 19:55:13.089667 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.089514 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 19:55:13.089667 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.089522 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 19:55:13.089667 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.089534 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 19:55:13.089667 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.089542 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 19:55:13.089667 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.089565 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 19:55:13.089667 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.089571 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 19:55:13.091123 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.091109 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 19:55:13.091123 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.091124 2548 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 19:55:13.094819 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.094802 2548 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 19:55:13.094897 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.094843 2548 server.go:1295] "Started kubelet" Jun 02 19:55:13.094961 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.094934 2548 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 19:55:13.094961 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.094930 2548 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 19:55:13.095049 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.094985 2548 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 19:55:13.097263 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.097237 2548 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 19:55:13.097300 ip-10-0-129-157 systemd[1]: Started Kubernetes Kubelet. Jun 02 19:55:13.099086 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.099067 2548 server.go:317] "Adding debug handlers to kubelet server" Jun 02 19:55:13.100397 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.100374 2548 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-157.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 19:55:13.100474 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.100441 2548 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 19:55:13.100526 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.100487 2548 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 19:55:13.103165 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.103149 2548 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 19:55:13.103248 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.103164 2548 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 19:55:13.103248 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.103207 2548 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 19:55:13.103248 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.103219 2548 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 19:55:13.103372 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.103319 2548 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 19:55:13.103414 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.103375 2548 reconstruct.go:97] "Volume reconstruction finished" Jun 02 19:55:13.103414 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.103383 2548 reconciler.go:26] "Reconciler: start to sync state" Jun 02 19:55:13.103898 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.103862 2548 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jun 02 19:55:13.104075 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.104058 2548 factory.go:55] Registering systemd factory Jun 02 19:55:13.104150 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.104081 2548 factory.go:223] Registration of the systemd container factory successfully Jun 02 19:55:13.104311 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.104295 2548 factory.go:153] Registering CRI-O factory Jun 02 19:55:13.104311 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.104308 2548 factory.go:223] Registration of the crio container factory successfully Jun 02 19:55:13.104482 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.104351 2548 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 19:55:13.104482 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.104367 2548 factory.go:103] Registering Raw factory Jun 02 19:55:13.104482 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.104382 2548 manager.go:1196] Started watching for new ooms in manager Jun 02 19:55:13.104844 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.104828 2548 manager.go:319] Starting recovery of all containers Jun 02 19:55:13.106117 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.105166 2548 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-157.ec2.internal.18b55bc1366ccfce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-157.ec2.internal,UID:ip-10-0-129-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-157.ec2.internal,},FirstTimestamp:2026-06-02 19:55:13.094815694 +0000 UTC m=+0.412028938,LastTimestamp:2026-06-02 19:55:13.094815694 +0000 UTC m=+0.412028938,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-157.ec2.internal,}" Jun 02 19:55:13.111674 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.107290 2548 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 19:55:13.112891 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.112871 2548 manager.go:324] Recovery completed Jun 02 19:55:13.117644 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.117595 2548 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 02 19:55:13.117726 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.117712 2548 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:55:13.118070 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.118050 2548 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 02 19:55:13.119684 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.119668 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:55:13.119766 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.119698 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:55:13.119766 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.119711 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:55:13.120249 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.120224 2548 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 19:55:13.120249 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.120237 2548 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 19:55:13.120411 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.120260 2548 state_mem.go:36] "Initialized new in-memory state store" Jun 02 19:55:13.122775 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.122757 2548 policy_none.go:49] "None policy: Start" Jun 02 19:55:13.122852 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.122782 2548 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 19:55:13.122905 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.122860 2548 state_mem.go:35] "Initializing new in-memory state store" Jun 02 19:55:13.124321 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.124215 2548 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-157.ec2.internal.18b55bc137e84b1e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-157.ec2.internal,UID:ip-10-0-129-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-157.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-157.ec2.internal,},FirstTimestamp:2026-06-02 19:55:13.119685406 +0000 UTC m=+0.436898651,LastTimestamp:2026-06-02 19:55:13.119685406 +0000 UTC m=+0.436898651,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-157.ec2.internal,}" Jun 02 19:55:13.138430 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.138345 2548 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-157.ec2.internal.18b55bc137e89366 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-157.ec2.internal,UID:ip-10-0-129-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-157.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-157.ec2.internal,},FirstTimestamp:2026-06-02 19:55:13.11970391 +0000 UTC m=+0.436917156,LastTimestamp:2026-06-02 19:55:13.11970391 +0000 UTC m=+0.436917156,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-157.ec2.internal,}" Jun 02 19:55:13.150135 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.150055 2548 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-157.ec2.internal.18b55bc137e8bcff default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-157.ec2.internal,UID:ip-10-0-129-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-157.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-157.ec2.internal,},FirstTimestamp:2026-06-02 19:55:13.119714559 +0000 UTC m=+0.436927804,LastTimestamp:2026-06-02 19:55:13.119714559 +0000 UTC m=+0.436927804,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-157.ec2.internal,}" Jun 02 19:55:13.164489 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.164466 2548 manager.go:341] "Starting Device Plugin manager" Jun 02 19:55:13.174427 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.164513 2548 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 19:55:13.174427 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.164528 2548 server.go:85] "Starting device plugin registration server" Jun 02 19:55:13.174427 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.164879 2548 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 19:55:13.174427 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.164891 2548 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 19:55:13.174427 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.165037 2548 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 19:55:13.174427 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.165123 2548 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 19:55:13.174427 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.165132 2548 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 19:55:13.174427 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.165946 2548 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 19:55:13.174427 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.165980 2548 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-157.ec2.internal\" not found" Jun 02 19:55:13.175485 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.175465 2548 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p2g8j" Jun 02 19:55:13.179135 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.179053 2548 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-157.ec2.internal.18b55bc13ab78427 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-157.ec2.internal,UID:ip-10-0-129-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-129-157.ec2.internal,},FirstTimestamp:2026-06-02 19:55:13.166820391 +0000 UTC m=+0.484033622,LastTimestamp:2026-06-02 19:55:13.166820391 +0000 UTC m=+0.484033622,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-157.ec2.internal,}" Jun 02 19:55:13.187642 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.187621 2548 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p2g8j" Jun 02 19:55:13.265351 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.265284 2548 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:55:13.266305 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.266285 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:55:13.266397 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.266318 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:55:13.266397 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.266330 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:55:13.266397 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.266355 2548 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.288261 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.288224 2548 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.288261 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.288261 2548 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-157.ec2.internal\": node \"ip-10-0-129-157.ec2.internal\" not found" Jun 02 19:55:13.302526 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.302498 2548 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jun 02 19:55:13.311638 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.311605 2548 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 19:55:13.312873 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.312853 2548 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 19:55:13.312978 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.312883 2548 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 19:55:13.312978 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.312902 2548 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 19:55:13.312978 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.312914 2548 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 19:55:13.312978 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.312947 2548 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 19:55:13.315796 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.315775 2548 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:55:13.403408 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.403355 2548 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jun 02 19:55:13.413715 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.413689 2548 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal"] Jun 02 19:55:13.413807 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.413802 2548 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:55:13.414812 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.414786 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:55:13.414910 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.414818 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:55:13.414910 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.414828 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:55:13.416144 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.416132 2548 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:55:13.416294 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.416281 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.416333 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.416313 2548 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:55:13.416864 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.416848 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:55:13.416938 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.416864 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:55:13.416938 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.416874 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:55:13.416938 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.416884 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:55:13.416938 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.416887 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:55:13.416938 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.416894 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:55:13.418011 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.417997 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.418060 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.418023 2548 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:55:13.418726 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.418709 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:55:13.418832 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.418733 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:55:13.418832 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.418760 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:55:13.444650 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.444617 2548 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-157.ec2.internal\" not found" node="ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.449369 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.449351 2548 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-157.ec2.internal\" not found" node="ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.504436 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.504404 2548 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jun 02 19:55:13.506635 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.506609 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2dcf4b3d1e27492b4997b358cd9a9f92-config\") pod \"kube-apiserver-proxy-ip-10-0-129-157.ec2.internal\" (UID: \"2dcf4b3d1e27492b4997b358cd9a9f92\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.506720 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.506650 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4bcb9fdd770434633007c1d0f5480ec1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal\" (UID: \"4bcb9fdd770434633007c1d0f5480ec1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.506720 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.506680 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4bcb9fdd770434633007c1d0f5480ec1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal\" (UID: \"4bcb9fdd770434633007c1d0f5480ec1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.605216 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.605136 2548 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jun 02 19:55:13.607334 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.607314 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2dcf4b3d1e27492b4997b358cd9a9f92-config\") pod \"kube-apiserver-proxy-ip-10-0-129-157.ec2.internal\" (UID: \"2dcf4b3d1e27492b4997b358cd9a9f92\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.607391 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.607344 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4bcb9fdd770434633007c1d0f5480ec1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal\" (UID: \"4bcb9fdd770434633007c1d0f5480ec1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.607391 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.607361 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4bcb9fdd770434633007c1d0f5480ec1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal\" (UID: \"4bcb9fdd770434633007c1d0f5480ec1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.607452 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.607415 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4bcb9fdd770434633007c1d0f5480ec1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal\" (UID: \"4bcb9fdd770434633007c1d0f5480ec1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.607452 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.607420 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4bcb9fdd770434633007c1d0f5480ec1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal\" (UID: \"4bcb9fdd770434633007c1d0f5480ec1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.607514 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.607464 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2dcf4b3d1e27492b4997b358cd9a9f92-config\") pod \"kube-apiserver-proxy-ip-10-0-129-157.ec2.internal\" (UID: \"2dcf4b3d1e27492b4997b358cd9a9f92\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.706239 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.706203 2548 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jun 02 19:55:13.746538 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.746497 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.753113 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:13.753091 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" Jun 02 19:55:13.806719 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.806678 2548 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jun 02 19:55:13.907219 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:13.907139 2548 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jun 02 19:55:14.007615 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:14.007584 2548 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jun 02 19:55:14.013898 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.013877 2548 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 19:55:14.014071 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.014049 2548 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 19:55:14.104224 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.104197 2548 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 19:55:14.108184 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:14.108167 2548 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jun 02 19:55:14.128223 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.128192 2548 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 19:55:14.174242 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.174148 2548 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wn9dg" Jun 02 19:55:14.189275 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.189233 2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 19:50:13 +0000 UTC" deadline="2027-10-27 11:38:13.950395769 +0000 UTC" Jun 02 19:55:14.189275 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.189273 2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12279h42m59.761125431s" Jun 02 19:55:14.193056 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.193040 2548 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wn9dg" Jun 02 19:55:14.204573 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.204551 2548 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:55:14.304232 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.304198 2548 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jun 02 19:55:14.329987 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.329961 2548 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 19:55:14.330953 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.330937 2548 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" Jun 02 19:55:14.343999 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.343979 2548 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 19:55:14.354005 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:14.353961 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bcb9fdd770434633007c1d0f5480ec1.slice/crio-2e6374cc8ec806e9e2b890dd5dfb07e048cbb4bf77e57cc21453290be0af9b23 WatchSource:0}: Error finding container 2e6374cc8ec806e9e2b890dd5dfb07e048cbb4bf77e57cc21453290be0af9b23: Status 404 returned error can't find the container with id 2e6374cc8ec806e9e2b890dd5dfb07e048cbb4bf77e57cc21453290be0af9b23 Jun 02 19:55:14.357029 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:14.357003 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dcf4b3d1e27492b4997b358cd9a9f92.slice/crio-a605b5886229bfd3c0c8d0ab581ebd20c6d22fcc6915b67675f7c90486c3bf45 WatchSource:0}: Error finding container a605b5886229bfd3c0c8d0ab581ebd20c6d22fcc6915b67675f7c90486c3bf45: Status 404 returned error can't find the container with id a605b5886229bfd3c0c8d0ab581ebd20c6d22fcc6915b67675f7c90486c3bf45 Jun 02 19:55:14.359518 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.359504 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 19:55:14.450135 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.449183 2548 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:55:14.618119 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:14.618089 2548 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:55:15.082352 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.082320 2548 apiserver.go:52] "Watching apiserver" Jun 02 19:55:15.096180 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.096149 2548 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 19:55:15.098317 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.098284 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-lz2mg","openshift-multus/multus-additional-cni-plugins-j5w6p","openshift-network-diagnostics/network-check-target-62ckd","openshift-network-operator/iptables-alerter-m4sth","openshift-ovn-kubernetes/ovnkube-node-np2n4","kube-system/konnectivity-agent-2ng7z","kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl","openshift-cluster-node-tuning-operator/tuned-nscs2","openshift-image-registry/node-ca-kcv4k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal","openshift-multus/multus-52r6t","openshift-multus/network-metrics-daemon-wwpfn"] Jun 02 19:55:15.100263 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.100239 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kcv4k" Jun 02 19:55:15.101774 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.101752 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.103164 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.103145 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:15.103256 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.103219 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:15.103621 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.103604 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 19:55:15.104134 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.104111 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 19:55:15.104230 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.104125 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 19:55:15.104230 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.104223 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lcsz4\"" Jun 02 19:55:15.105977 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.105957 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m4sth" Jun 02 19:55:15.106453 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.106434 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 19:55:15.106942 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.106827 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 19:55:15.106942 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.106919 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 19:55:15.107099 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.106924 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 19:55:15.107218 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.107203 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hsrv6\"" Jun 02 19:55:15.107355 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.107336 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 19:55:15.107928 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.107733 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.110535 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.110515 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:55:15.110900 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.110881 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2ng7z" Jun 02 19:55:15.111359 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.111108 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.111827 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.111594 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 19:55:15.111827 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.111603 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-74zj6\"" Jun 02 19:55:15.111827 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.111695 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 19:55:15.112431 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.112413 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.114266 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.114247 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lz2mg" Jun 02 19:55:15.114694 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.114669 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4271c2ac-658b-4949-b178-ac6530158698-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.114820 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.114713 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4271c2ac-658b-4949-b178-ac6530158698-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.114820 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.114760 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/861f1595-8ced-49bc-b4e3-2db727f3bb25-host\") pod \"node-ca-kcv4k\" (UID: \"861f1595-8ced-49bc-b4e3-2db727f3bb25\") " pod="openshift-image-registry/node-ca-kcv4k" Jun 02 19:55:15.114820 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.114786 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkjnv\" (UniqueName: \"kubernetes.io/projected/861f1595-8ced-49bc-b4e3-2db727f3bb25-kube-api-access-nkjnv\") pod \"node-ca-kcv4k\" (UID: \"861f1595-8ced-49bc-b4e3-2db727f3bb25\") " pod="openshift-image-registry/node-ca-kcv4k" Jun 02 19:55:15.114820 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.114810 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4271c2ac-658b-4949-b178-ac6530158698-cnibin\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.115022 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.114833 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4271c2ac-658b-4949-b178-ac6530158698-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.115022 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.114882 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4jqz\" (UniqueName: \"kubernetes.io/projected/4271c2ac-658b-4949-b178-ac6530158698-kube-api-access-x4jqz\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.115022 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.114919 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4271c2ac-658b-4949-b178-ac6530158698-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.115022 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.114946 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/990f9600-65e9-4e58-80e8-862601023a1a-iptables-alerter-script\") pod \"iptables-alerter-m4sth\" (UID: \"990f9600-65e9-4e58-80e8-862601023a1a\") " pod="openshift-network-operator/iptables-alerter-m4sth" Jun 02 19:55:15.115022 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.114967 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/990f9600-65e9-4e58-80e8-862601023a1a-host-slash\") pod \"iptables-alerter-m4sth\" (UID: \"990f9600-65e9-4e58-80e8-862601023a1a\") " pod="openshift-network-operator/iptables-alerter-m4sth" Jun 02 19:55:15.115022 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.114992 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4271c2ac-658b-4949-b178-ac6530158698-system-cni-dir\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.115022 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.115015 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4271c2ac-658b-4949-b178-ac6530158698-os-release\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.115360 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.115051 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bqvj\" (UniqueName: \"kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj\") pod \"network-check-target-62ckd\" (UID: \"ea07bb39-2258-40a3-bc5f-a05dc1e90164\") " pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:15.115360 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.115107 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj5lx\" (UniqueName: \"kubernetes.io/projected/990f9600-65e9-4e58-80e8-862601023a1a-kube-api-access-rj5lx\") pod \"iptables-alerter-m4sth\" (UID: \"990f9600-65e9-4e58-80e8-862601023a1a\") " pod="openshift-network-operator/iptables-alerter-m4sth" Jun 02 19:55:15.115360 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.115140 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/861f1595-8ced-49bc-b4e3-2db727f3bb25-serviceca\") pod \"node-ca-kcv4k\" (UID: \"861f1595-8ced-49bc-b4e3-2db727f3bb25\") " pod="openshift-image-registry/node-ca-kcv4k" Jun 02 19:55:15.115567 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.115548 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.117836 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.117808 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:15.117925 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.117877 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:15.118688 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.118660 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 19:55:15.118804 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.118679 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 19:55:15.122809 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.120134 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 19:55:15.122809 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.120197 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m64s4\"" Jun 02 19:55:15.122809 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.120336 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 19:55:15.122809 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.120475 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bwnts\"" Jun 02 19:55:15.122809 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.120521 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 19:55:15.122809 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.120532 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 19:55:15.122809 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.120589 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 19:55:15.122809 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.121233 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 19:55:15.122809 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.121454 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 19:55:15.123269 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.123026 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 19:55:15.123269 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.123081 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 19:55:15.123269 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.123172 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6c5c9\"" Jun 02 19:55:15.123654 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.123434 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:55:15.123654 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.123441 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 19:55:15.123654 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.123501 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-shsvm\"" Jun 02 19:55:15.123654 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.123506 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 19:55:15.123654 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.123530 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 19:55:15.124598 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.124579 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 19:55:15.124696 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.124638 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4fqsj\"" Jun 02 19:55:15.124696 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.124670 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zgnzk\"" Jun 02 19:55:15.195097 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.195050 2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 19:50:14 +0000 UTC" deadline="2027-12-25 22:43:05.866520609 +0000 UTC" Jun 02 19:55:15.195097 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.195089 2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13706h47m50.671435282s" Jun 02 19:55:15.205151 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.205122 2548 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 19:55:15.215952 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.215920 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-var-lib-cni-bin\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.216151 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.215963 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-kubelet\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.216151 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.215988 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-systemd\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.216151 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216019 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/990f9600-65e9-4e58-80e8-862601023a1a-host-slash\") pod \"iptables-alerter-m4sth\" (UID: \"990f9600-65e9-4e58-80e8-862601023a1a\") " pod="openshift-network-operator/iptables-alerter-m4sth" Jun 02 19:55:15.216151 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216046 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4271c2ac-658b-4949-b178-ac6530158698-os-release\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.216151 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216082 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-run-multus-certs\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.216151 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216107 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-multus-conf-dir\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.216151 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216133 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57n5f\" (UniqueName: \"kubernetes.io/projected/372120bf-121b-4dff-b2b0-f89a944a42dc-kube-api-access-57n5f\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.216481 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216160 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:15.216481 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216188 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4bqvj\" (UniqueName: \"kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj\") pod \"network-check-target-62ckd\" (UID: \"ea07bb39-2258-40a3-bc5f-a05dc1e90164\") " pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:15.216481 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216212 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/990f9600-65e9-4e58-80e8-862601023a1a-host-slash\") pod \"iptables-alerter-m4sth\" (UID: \"990f9600-65e9-4e58-80e8-862601023a1a\") " pod="openshift-network-operator/iptables-alerter-m4sth" Jun 02 19:55:15.216481 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216215 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rj5lx\" (UniqueName: \"kubernetes.io/projected/990f9600-65e9-4e58-80e8-862601023a1a-kube-api-access-rj5lx\") pod \"iptables-alerter-m4sth\" (UID: \"990f9600-65e9-4e58-80e8-862601023a1a\") " pod="openshift-network-operator/iptables-alerter-m4sth" Jun 02 19:55:15.216481 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216319 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4271c2ac-658b-4949-b178-ac6530158698-os-release\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.216481 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216392 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-cnibin\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.216481 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216453 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-run-netns\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.216792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216490 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-log-socket\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.216792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216517 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcbbef18-c454-4403-861d-a1cd4cae3ce9-ovnkube-script-lib\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.216792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216549 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nkjnv\" (UniqueName: \"kubernetes.io/projected/861f1595-8ced-49bc-b4e3-2db727f3bb25-kube-api-access-nkjnv\") pod \"node-ca-kcv4k\" (UID: \"861f1595-8ced-49bc-b4e3-2db727f3bb25\") " pod="openshift-image-registry/node-ca-kcv4k" Jun 02 19:55:15.216792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216575 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4271c2ac-658b-4949-b178-ac6530158698-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.216792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216602 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/286e8706-b9e4-4ae9-b67f-3ddbcf5877a6-agent-certs\") pod \"konnectivity-agent-2ng7z\" (UID: \"286e8706-b9e4-4ae9-b67f-3ddbcf5877a6\") " pod="kube-system/konnectivity-agent-2ng7z" Jun 02 19:55:15.216792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216626 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-run\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.216792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216650 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ncl7\" (UniqueName: \"kubernetes.io/projected/6c422453-6b93-4de0-b000-13511f092b3f-kube-api-access-4ncl7\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.216792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216675 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/286e8706-b9e4-4ae9-b67f-3ddbcf5877a6-konnectivity-ca\") pod \"konnectivity-agent-2ng7z\" (UID: \"286e8706-b9e4-4ae9-b67f-3ddbcf5877a6\") " pod="kube-system/konnectivity-agent-2ng7z" Jun 02 19:55:15.216792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216700 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-var-lib-cni-multus\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.216792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216754 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/372120bf-121b-4dff-b2b0-f89a944a42dc-cni-binary-copy\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.216792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216779 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4271c2ac-658b-4949-b178-ac6530158698-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.216792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216791 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9mrq\" (UniqueName: \"kubernetes.io/projected/c848a2cb-1edf-4aa2-b593-c0f19d847476-kube-api-access-b9mrq\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216838 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-etc-selinux\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216883 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-modprobe-d\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216906 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-host\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.216942 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4271c2ac-658b-4949-b178-ac6530158698-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217020 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-cni-bin\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217047 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcbbef18-c454-4403-861d-a1cd4cae3ce9-ovnkube-config\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217084 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-registration-dir\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217113 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c3674c37-2e21-4799-9545-d69c1985b721-hosts-file\") pod \"node-resolver-lz2mg\" (UID: \"c3674c37-2e21-4799-9545-d69c1985b721\") " pod="openshift-dns/node-resolver-lz2mg" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217137 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4271c2ac-658b-4949-b178-ac6530158698-cnibin\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217159 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-run-k8s-cni-cncf-io\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217183 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcbbef18-c454-4403-861d-a1cd4cae3ce9-env-overrides\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217206 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4271c2ac-658b-4949-b178-ac6530158698-cnibin\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217219 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-var-lib-kubelet\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217257 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4271c2ac-658b-4949-b178-ac6530158698-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.217289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217284 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-run-netns\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217307 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-etc-kubernetes\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217330 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-var-lib-openvswitch\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217372 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-run-openvswitch\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217403 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217428 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcbbef18-c454-4403-861d-a1cd4cae3ce9-ovn-node-metrics-cert\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217479 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtxpw\" (UniqueName: \"kubernetes.io/projected/14272226-bd37-4c8c-94e5-221e0fd96c1c-kube-api-access-wtxpw\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217503 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-node-log\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217554 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-device-dir\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217595 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-sysconfig\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217621 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-kubernetes\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217649 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-sysctl-d\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217657 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4271c2ac-658b-4949-b178-ac6530158698-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217686 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-sysctl-conf\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217718 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-sys\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217754 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3674c37-2e21-4799-9545-d69c1985b721-tmp-dir\") pod \"node-resolver-lz2mg\" (UID: \"c3674c37-2e21-4799-9545-d69c1985b721\") " pod="openshift-dns/node-resolver-lz2mg" Jun 02 19:55:15.218064 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217780 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/861f1595-8ced-49bc-b4e3-2db727f3bb25-serviceca\") pod \"node-ca-kcv4k\" (UID: \"861f1595-8ced-49bc-b4e3-2db727f3bb25\") " pod="openshift-image-registry/node-ca-kcv4k" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217814 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4271c2ac-658b-4949-b178-ac6530158698-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217860 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4271c2ac-658b-4949-b178-ac6530158698-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217893 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-multus-cni-dir\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217917 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-os-release\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217941 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-var-lib-kubelet\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217964 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/372120bf-121b-4dff-b2b0-f89a944a42dc-multus-daemon-config\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.217993 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-etc-openvswitch\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218016 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-socket-dir\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218042 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/861f1595-8ced-49bc-b4e3-2db727f3bb25-host\") pod \"node-ca-kcv4k\" (UID: \"861f1595-8ced-49bc-b4e3-2db727f3bb25\") " pod="openshift-image-registry/node-ca-kcv4k" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218069 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x4jqz\" (UniqueName: \"kubernetes.io/projected/4271c2ac-658b-4949-b178-ac6530158698-kube-api-access-x4jqz\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218096 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-system-cni-dir\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218121 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-slash\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218146 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-sys-fs\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218170 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c422453-6b93-4de0-b000-13511f092b3f-etc-tuned\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218197 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x9rl\" (UniqueName: \"kubernetes.io/projected/c3674c37-2e21-4799-9545-d69c1985b721-kube-api-access-6x9rl\") pod \"node-resolver-lz2mg\" (UID: \"c3674c37-2e21-4799-9545-d69c1985b721\") " pod="openshift-dns/node-resolver-lz2mg" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218225 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/990f9600-65e9-4e58-80e8-862601023a1a-iptables-alerter-script\") pod \"iptables-alerter-m4sth\" (UID: \"990f9600-65e9-4e58-80e8-862601023a1a\") " pod="openshift-network-operator/iptables-alerter-m4sth" Jun 02 19:55:15.218791 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218250 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/861f1595-8ced-49bc-b4e3-2db727f3bb25-serviceca\") pod \"node-ca-kcv4k\" (UID: \"861f1595-8ced-49bc-b4e3-2db727f3bb25\") " pod="openshift-image-registry/node-ca-kcv4k" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218254 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4271c2ac-658b-4949-b178-ac6530158698-system-cni-dir\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218310 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-hostroot\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218336 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-run-ovn-kubernetes\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218339 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4271c2ac-658b-4949-b178-ac6530158698-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218363 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-cni-netd\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218392 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-lib-modules\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218398 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4271c2ac-658b-4949-b178-ac6530158698-system-cni-dir\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218418 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-multus-socket-dir-parent\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218469 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-run-systemd\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218472 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/861f1595-8ced-49bc-b4e3-2db727f3bb25-host\") pod \"node-ca-kcv4k\" (UID: \"861f1595-8ced-49bc-b4e3-2db727f3bb25\") " pod="openshift-image-registry/node-ca-kcv4k" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218517 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-489q4\" (UniqueName: \"kubernetes.io/projected/fcbbef18-c454-4403-861d-a1cd4cae3ce9-kube-api-access-489q4\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218585 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218725 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c422453-6b93-4de0-b000-13511f092b3f-tmp\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218783 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-systemd-units\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.218822 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-run-ovn\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.219568 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.219017 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/990f9600-65e9-4e58-80e8-862601023a1a-iptables-alerter-script\") pod \"iptables-alerter-m4sth\" (UID: \"990f9600-65e9-4e58-80e8-862601023a1a\") " pod="openshift-network-operator/iptables-alerter-m4sth" Jun 02 19:55:15.237624 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.237579 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:55:15.237624 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.237623 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:55:15.237894 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.237637 2548 projected.go:194] Error preparing data for projected volume kube-api-access-4bqvj for pod openshift-network-diagnostics/network-check-target-62ckd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:15.237894 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.237735 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj podName:ea07bb39-2258-40a3-bc5f-a05dc1e90164 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:15.737694655 +0000 UTC m=+3.054907887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4bqvj" (UniqueName: "kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj") pod "network-check-target-62ckd" (UID: "ea07bb39-2258-40a3-bc5f-a05dc1e90164") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:15.243315 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.243281 2548 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 19:55:15.247107 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.246890 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkjnv\" (UniqueName: \"kubernetes.io/projected/861f1595-8ced-49bc-b4e3-2db727f3bb25-kube-api-access-nkjnv\") pod \"node-ca-kcv4k\" (UID: \"861f1595-8ced-49bc-b4e3-2db727f3bb25\") " pod="openshift-image-registry/node-ca-kcv4k" Jun 02 19:55:15.247248 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.246892 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4jqz\" (UniqueName: \"kubernetes.io/projected/4271c2ac-658b-4949-b178-ac6530158698-kube-api-access-x4jqz\") pod \"multus-additional-cni-plugins-j5w6p\" (UID: \"4271c2ac-658b-4949-b178-ac6530158698\") " pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.247248 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.247090 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj5lx\" (UniqueName: \"kubernetes.io/projected/990f9600-65e9-4e58-80e8-862601023a1a-kube-api-access-rj5lx\") pod \"iptables-alerter-m4sth\" (UID: \"990f9600-65e9-4e58-80e8-862601023a1a\") " pod="openshift-network-operator/iptables-alerter-m4sth" Jun 02 19:55:15.317306 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.317252 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" event={"ID":"2dcf4b3d1e27492b4997b358cd9a9f92","Type":"ContainerStarted","Data":"a605b5886229bfd3c0c8d0ab581ebd20c6d22fcc6915b67675f7c90486c3bf45"} Jun 02 19:55:15.318488 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.318462 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" event={"ID":"4bcb9fdd770434633007c1d0f5480ec1","Type":"ContainerStarted","Data":"2e6374cc8ec806e9e2b890dd5dfb07e048cbb4bf77e57cc21453290be0af9b23"} Jun 02 19:55:15.320713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320107 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c422453-6b93-4de0-b000-13511f092b3f-etc-tuned\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.320713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320285 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6x9rl\" (UniqueName: \"kubernetes.io/projected/c3674c37-2e21-4799-9545-d69c1985b721-kube-api-access-6x9rl\") pod \"node-resolver-lz2mg\" (UID: \"c3674c37-2e21-4799-9545-d69c1985b721\") " pod="openshift-dns/node-resolver-lz2mg" Jun 02 19:55:15.320713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320344 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-hostroot\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.320713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320379 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-run-ovn-kubernetes\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.320713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320408 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-cni-netd\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.320713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320438 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-lib-modules\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.320713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320468 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-multus-socket-dir-parent\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.320713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320495 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-run-systemd\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.320713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320528 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-489q4\" (UniqueName: \"kubernetes.io/projected/fcbbef18-c454-4403-861d-a1cd4cae3ce9-kube-api-access-489q4\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.320713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320560 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.320713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320657 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-run-systemd\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.320713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320662 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-hostroot\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.321390 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320764 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-multus-socket-dir-parent\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.321390 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320789 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-run-ovn-kubernetes\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.321390 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320848 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-cni-netd\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.321390 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320924 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-lib-modules\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.321390 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.320992 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.321390 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321104 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c422453-6b93-4de0-b000-13511f092b3f-tmp\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.321390 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321175 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-systemd-units\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.321390 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321207 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-run-ovn\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.321390 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321238 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-var-lib-cni-bin\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.321390 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321262 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-kubelet\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.321390 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321292 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-systemd\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321406 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-run-multus-certs\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321437 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-multus-conf-dir\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321466 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-57n5f\" (UniqueName: \"kubernetes.io/projected/372120bf-121b-4dff-b2b0-f89a944a42dc-kube-api-access-57n5f\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321504 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321548 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-cnibin\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321589 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-run-netns\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321619 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-log-socket\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321648 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcbbef18-c454-4403-861d-a1cd4cae3ce9-ovnkube-script-lib\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321678 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-systemd\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321801 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/286e8706-b9e4-4ae9-b67f-3ddbcf5877a6-agent-certs\") pod \"konnectivity-agent-2ng7z\" (UID: \"286e8706-b9e4-4ae9-b67f-3ddbcf5877a6\") " pod="kube-system/konnectivity-agent-2ng7z" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321839 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-run\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321861 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-run-ovn\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.321927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321869 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4ncl7\" (UniqueName: \"kubernetes.io/projected/6c422453-6b93-4de0-b000-13511f092b3f-kube-api-access-4ncl7\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.322502 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.321942 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-systemd-units\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.322502 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322009 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-run-netns\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.322502 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322061 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-run-multus-certs\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.322502 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322108 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-multus-conf-dir\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.322502 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322240 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-log-socket\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.322732 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322582 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-run\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.322732 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322652 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-cnibin\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.322732 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322686 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/286e8706-b9e4-4ae9-b67f-3ddbcf5877a6-konnectivity-ca\") pod \"konnectivity-agent-2ng7z\" (UID: \"286e8706-b9e4-4ae9-b67f-3ddbcf5877a6\") " pod="kube-system/konnectivity-agent-2ng7z" Jun 02 19:55:15.322949 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322734 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-var-lib-cni-multus\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.322949 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322799 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/372120bf-121b-4dff-b2b0-f89a944a42dc-cni-binary-copy\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.322949 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322857 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b9mrq\" (UniqueName: \"kubernetes.io/projected/c848a2cb-1edf-4aa2-b593-c0f19d847476-kube-api-access-b9mrq\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:15.322949 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322912 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-etc-selinux\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.323136 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322954 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-modprobe-d\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.323136 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.322993 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-host\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.323136 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323018 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-cni-bin\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.323136 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323079 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcbbef18-c454-4403-861d-a1cd4cae3ce9-ovnkube-config\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.323315 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323141 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcbbef18-c454-4403-861d-a1cd4cae3ce9-ovnkube-script-lib\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.323315 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323247 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-registration-dir\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.323405 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.323354 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:15.323564 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.323550 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs podName:c848a2cb-1edf-4aa2-b593-c0f19d847476 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:15.823524312 +0000 UTC m=+3.140737557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs") pod "network-metrics-daemon-wwpfn" (UID: "c848a2cb-1edf-4aa2-b593-c0f19d847476") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:15.323624 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323133 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-registration-dir\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323715 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/286e8706-b9e4-4ae9-b67f-3ddbcf5877a6-konnectivity-ca\") pod \"konnectivity-agent-2ng7z\" (UID: \"286e8706-b9e4-4ae9-b67f-3ddbcf5877a6\") " pod="kube-system/konnectivity-agent-2ng7z" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323728 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-kubelet\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323815 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-modprobe-d\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323843 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-var-lib-cni-bin\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323916 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-etc-selinux\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323944 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-host\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323954 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c3674c37-2e21-4799-9545-d69c1985b721-hosts-file\") pod \"node-resolver-lz2mg\" (UID: \"c3674c37-2e21-4799-9545-d69c1985b721\") " pod="openshift-dns/node-resolver-lz2mg" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.323989 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-var-lib-cni-multus\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324000 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-run-k8s-cni-cncf-io\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324032 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcbbef18-c454-4403-861d-a1cd4cae3ce9-env-overrides\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324042 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-cni-bin\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324121 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-var-lib-kubelet\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324183 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-run-k8s-cni-cncf-io\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324189 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-run-netns\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324348 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-run-netns\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324351 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/372120bf-121b-4dff-b2b0-f89a944a42dc-cni-binary-copy\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324383 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-etc-kubernetes\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.324579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324431 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c3674c37-2e21-4799-9545-d69c1985b721-hosts-file\") pod \"node-resolver-lz2mg\" (UID: \"c3674c37-2e21-4799-9545-d69c1985b721\") " pod="openshift-dns/node-resolver-lz2mg" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324458 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-var-lib-openvswitch\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324479 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-var-lib-kubelet\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324495 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-run-openvswitch\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324559 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324594 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcbbef18-c454-4403-861d-a1cd4cae3ce9-ovn-node-metrics-cert\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324628 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wtxpw\" (UniqueName: \"kubernetes.io/projected/14272226-bd37-4c8c-94e5-221e0fd96c1c-kube-api-access-wtxpw\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324660 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcbbef18-c454-4403-861d-a1cd4cae3ce9-ovnkube-config\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324694 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcbbef18-c454-4403-861d-a1cd4cae3ce9-env-overrides\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324718 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-run-openvswitch\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324762 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-etc-kubernetes\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324792 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-var-lib-openvswitch\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324684 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-node-log\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324887 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-device-dir\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324957 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-sysconfig\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324961 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.324958 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-device-dir\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.325508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325012 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-kubernetes\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325042 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-sysconfig\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325045 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-sysctl-d\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325084 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-sysctl-conf\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325138 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-sys\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325091 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-kubernetes\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325280 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-sysctl-d\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325321 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3674c37-2e21-4799-9545-d69c1985b721-tmp-dir\") pod \"node-resolver-lz2mg\" (UID: \"c3674c37-2e21-4799-9545-d69c1985b721\") " pod="openshift-dns/node-resolver-lz2mg" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325380 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-sys\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325495 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c422453-6b93-4de0-b000-13511f092b3f-etc-sysctl-conf\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325620 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-multus-cni-dir\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325750 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-node-log\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325872 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3674c37-2e21-4799-9545-d69c1985b721-tmp-dir\") pod \"node-resolver-lz2mg\" (UID: \"c3674c37-2e21-4799-9545-d69c1985b721\") " pod="openshift-dns/node-resolver-lz2mg" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325942 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c422453-6b93-4de0-b000-13511f092b3f-tmp\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.325998 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c422453-6b93-4de0-b000-13511f092b3f-etc-tuned\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326020 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-os-release\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326021 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-os-release\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326076 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-var-lib-kubelet\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.326288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326110 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/372120bf-121b-4dff-b2b0-f89a944a42dc-multus-daemon-config\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.327202 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326149 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-etc-openvswitch\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.327202 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326177 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-socket-dir\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.327202 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326213 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-system-cni-dir\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.327202 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326213 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-multus-cni-dir\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.327202 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326248 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-slash\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.327202 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326281 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-sys-fs\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.327202 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326307 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-etc-openvswitch\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.327202 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326472 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-sys-fs\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.327202 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326588 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14272226-bd37-4c8c-94e5-221e0fd96c1c-socket-dir\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.327202 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326644 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-system-cni-dir\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.327202 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326153 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/372120bf-121b-4dff-b2b0-f89a944a42dc-host-var-lib-kubelet\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.327202 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.326709 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcbbef18-c454-4403-861d-a1cd4cae3ce9-host-slash\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.327783 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.327484 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/372120bf-121b-4dff-b2b0-f89a944a42dc-multus-daemon-config\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.328060 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.328032 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/286e8706-b9e4-4ae9-b67f-3ddbcf5877a6-agent-certs\") pod \"konnectivity-agent-2ng7z\" (UID: \"286e8706-b9e4-4ae9-b67f-3ddbcf5877a6\") " pod="kube-system/konnectivity-agent-2ng7z" Jun 02 19:55:15.328060 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.328072 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcbbef18-c454-4403-861d-a1cd4cae3ce9-ovn-node-metrics-cert\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.349585 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.349519 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x9rl\" (UniqueName: \"kubernetes.io/projected/c3674c37-2e21-4799-9545-d69c1985b721-kube-api-access-6x9rl\") pod \"node-resolver-lz2mg\" (UID: \"c3674c37-2e21-4799-9545-d69c1985b721\") " pod="openshift-dns/node-resolver-lz2mg" Jun 02 19:55:15.351699 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.351672 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9mrq\" (UniqueName: \"kubernetes.io/projected/c848a2cb-1edf-4aa2-b593-c0f19d847476-kube-api-access-b9mrq\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:15.352479 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.352423 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtxpw\" (UniqueName: \"kubernetes.io/projected/14272226-bd37-4c8c-94e5-221e0fd96c1c-kube-api-access-wtxpw\") pod \"aws-ebs-csi-driver-node-d7hnl\" (UID: \"14272226-bd37-4c8c-94e5-221e0fd96c1c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.352600 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.352577 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57n5f\" (UniqueName: \"kubernetes.io/projected/372120bf-121b-4dff-b2b0-f89a944a42dc-kube-api-access-57n5f\") pod \"multus-52r6t\" (UID: \"372120bf-121b-4dff-b2b0-f89a944a42dc\") " pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.353492 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.353469 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-489q4\" (UniqueName: \"kubernetes.io/projected/fcbbef18-c454-4403-861d-a1cd4cae3ce9-kube-api-access-489q4\") pod \"ovnkube-node-np2n4\" (UID: \"fcbbef18-c454-4403-861d-a1cd4cae3ce9\") " pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.354292 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.354275 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ncl7\" (UniqueName: \"kubernetes.io/projected/6c422453-6b93-4de0-b000-13511f092b3f-kube-api-access-4ncl7\") pod \"tuned-nscs2\" (UID: \"6c422453-6b93-4de0-b000-13511f092b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.412011 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.411972 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kcv4k" Jun 02 19:55:15.419838 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.419805 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j5w6p" Jun 02 19:55:15.431221 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.431198 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m4sth" Jun 02 19:55:15.437891 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.437869 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:15.445588 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.445563 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2ng7z" Jun 02 19:55:15.454274 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.454232 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" Jun 02 19:55:15.459038 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.459002 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nscs2" Jun 02 19:55:15.464641 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.464617 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lz2mg" Jun 02 19:55:15.471359 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.471323 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-52r6t" Jun 02 19:55:15.486279 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.486241 2548 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:55:15.830047 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.829956 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:15.830047 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:15.829996 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4bqvj\" (UniqueName: \"kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj\") pod \"network-check-target-62ckd\" (UID: \"ea07bb39-2258-40a3-bc5f-a05dc1e90164\") " pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:15.830249 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.830115 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:15.830249 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.830187 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs podName:c848a2cb-1edf-4aa2-b593-c0f19d847476 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:16.830168189 +0000 UTC m=+4.147381421 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs") pod "network-metrics-daemon-wwpfn" (UID: "c848a2cb-1edf-4aa2-b593-c0f19d847476") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:15.830249 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.830117 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:55:15.830249 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.830234 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:55:15.830249 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.830249 2548 projected.go:194] Error preparing data for projected volume kube-api-access-4bqvj for pod openshift-network-diagnostics/network-check-target-62ckd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:15.830444 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:15.830300 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj podName:ea07bb39-2258-40a3-bc5f-a05dc1e90164 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:16.830288892 +0000 UTC m=+4.147502124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bqvj" (UniqueName: "kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj") pod "network-check-target-62ckd" (UID: "ea07bb39-2258-40a3-bc5f-a05dc1e90164") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:16.196185 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.196028 2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 19:50:14 +0000 UTC" deadline="2027-12-02 23:40:55.425598858 +0000 UTC" Jun 02 19:55:16.196185 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.196058 2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13155h45m39.22954304s" Jun 02 19:55:16.204450 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:16.204421 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861f1595_8ced_49bc_b4e3_2db727f3bb25.slice/crio-c0cf282ca91e28664c7432f710f0cb8b0851d03d27b230fc3804abe37ac2e48d WatchSource:0}: Error finding container c0cf282ca91e28664c7432f710f0cb8b0851d03d27b230fc3804abe37ac2e48d: Status 404 returned error can't find the container with id c0cf282ca91e28664c7432f710f0cb8b0851d03d27b230fc3804abe37ac2e48d Jun 02 19:55:16.215712 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:16.215686 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod990f9600_65e9_4e58_80e8_862601023a1a.slice/crio-61a5a2c9a90c110284517badc65e5abfc9ad37f8da8a2f6b4f0317cdbee4ad31 WatchSource:0}: Error finding container 61a5a2c9a90c110284517badc65e5abfc9ad37f8da8a2f6b4f0317cdbee4ad31: Status 404 returned error can't find the container with id 61a5a2c9a90c110284517badc65e5abfc9ad37f8da8a2f6b4f0317cdbee4ad31 Jun 02 19:55:16.220164 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:16.220072 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c422453_6b93_4de0_b000_13511f092b3f.slice/crio-dc2360fd19f9c6181b29ddf154d44b71d172f254b9b3e1c889d9a6cfc62f3c94 WatchSource:0}: Error finding container dc2360fd19f9c6181b29ddf154d44b71d172f254b9b3e1c889d9a6cfc62f3c94: Status 404 returned error can't find the container with id dc2360fd19f9c6181b29ddf154d44b71d172f254b9b3e1c889d9a6cfc62f3c94 Jun 02 19:55:16.221781 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:16.221758 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcbbef18_c454_4403_861d_a1cd4cae3ce9.slice/crio-10992e1f4787cfecfa610ba655282dbda995caa352568fae407cc4206db83dfd WatchSource:0}: Error finding container 10992e1f4787cfecfa610ba655282dbda995caa352568fae407cc4206db83dfd: Status 404 returned error can't find the container with id 10992e1f4787cfecfa610ba655282dbda995caa352568fae407cc4206db83dfd Jun 02 19:55:16.224143 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:16.224113 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372120bf_121b_4dff_b2b0_f89a944a42dc.slice/crio-211f83adf33a301736810debec1c51c3d4788d5fc3af9b17b1d68c09a2483001 WatchSource:0}: Error finding container 211f83adf33a301736810debec1c51c3d4788d5fc3af9b17b1d68c09a2483001: Status 404 returned error can't find the container with id 211f83adf33a301736810debec1c51c3d4788d5fc3af9b17b1d68c09a2483001 Jun 02 19:55:16.224987 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:16.224938 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4271c2ac_658b_4949_b178_ac6530158698.slice/crio-a6f4f4d6f0a7cc2a561c6b2ee2ac337600ef898188eebdc6e2b8d5496ac76b3f WatchSource:0}: Error finding container a6f4f4d6f0a7cc2a561c6b2ee2ac337600ef898188eebdc6e2b8d5496ac76b3f: Status 404 returned error can't find the container with id a6f4f4d6f0a7cc2a561c6b2ee2ac337600ef898188eebdc6e2b8d5496ac76b3f Jun 02 19:55:16.226569 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:16.226474 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14272226_bd37_4c8c_94e5_221e0fd96c1c.slice/crio-70ab75900705d5e825c545d922ab923b494f953c2ef93a94356dbad97fe891e0 WatchSource:0}: Error finding container 70ab75900705d5e825c545d922ab923b494f953c2ef93a94356dbad97fe891e0: Status 404 returned error can't find the container with id 70ab75900705d5e825c545d922ab923b494f953c2ef93a94356dbad97fe891e0 Jun 02 19:55:16.227999 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:16.227977 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3674c37_2e21_4799_9545_d69c1985b721.slice/crio-4d16b215ef10667e3dc09c5064e6ab821fb7b308f879e0e625d203583c99216d WatchSource:0}: Error finding container 4d16b215ef10667e3dc09c5064e6ab821fb7b308f879e0e625d203583c99216d: Status 404 returned error can't find the container with id 4d16b215ef10667e3dc09c5064e6ab821fb7b308f879e0e625d203583c99216d Jun 02 19:55:16.228963 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:16.228943 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod286e8706_b9e4_4ae9_b67f_3ddbcf5877a6.slice/crio-cbe7366f3f10f39e6ab833ca538259a85ec7c74ccc0c1ef2ff00f9693522408d WatchSource:0}: Error finding container cbe7366f3f10f39e6ab833ca538259a85ec7c74ccc0c1ef2ff00f9693522408d: Status 404 returned error can't find the container with id cbe7366f3f10f39e6ab833ca538259a85ec7c74ccc0c1ef2ff00f9693522408d Jun 02 19:55:16.321509 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.321468 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" event={"ID":"fcbbef18-c454-4403-861d-a1cd4cae3ce9","Type":"ContainerStarted","Data":"10992e1f4787cfecfa610ba655282dbda995caa352568fae407cc4206db83dfd"} Jun 02 19:55:16.322671 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.322632 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m4sth" event={"ID":"990f9600-65e9-4e58-80e8-862601023a1a","Type":"ContainerStarted","Data":"61a5a2c9a90c110284517badc65e5abfc9ad37f8da8a2f6b4f0317cdbee4ad31"} Jun 02 19:55:16.323560 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.323535 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kcv4k" event={"ID":"861f1595-8ced-49bc-b4e3-2db727f3bb25","Type":"ContainerStarted","Data":"c0cf282ca91e28664c7432f710f0cb8b0851d03d27b230fc3804abe37ac2e48d"} Jun 02 19:55:16.324964 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.324941 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" event={"ID":"2dcf4b3d1e27492b4997b358cd9a9f92","Type":"ContainerStarted","Data":"ebc2e628d36369902a6b9115776fc97d4765fe7ad57e818679b66f804f6c8300"} Jun 02 19:55:16.325933 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.325910 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2ng7z" event={"ID":"286e8706-b9e4-4ae9-b67f-3ddbcf5877a6","Type":"ContainerStarted","Data":"cbe7366f3f10f39e6ab833ca538259a85ec7c74ccc0c1ef2ff00f9693522408d"} Jun 02 19:55:16.327147 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.327126 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" event={"ID":"14272226-bd37-4c8c-94e5-221e0fd96c1c","Type":"ContainerStarted","Data":"70ab75900705d5e825c545d922ab923b494f953c2ef93a94356dbad97fe891e0"} Jun 02 19:55:16.328133 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.328115 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5w6p" event={"ID":"4271c2ac-658b-4949-b178-ac6530158698","Type":"ContainerStarted","Data":"a6f4f4d6f0a7cc2a561c6b2ee2ac337600ef898188eebdc6e2b8d5496ac76b3f"} Jun 02 19:55:16.329074 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.329052 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-52r6t" event={"ID":"372120bf-121b-4dff-b2b0-f89a944a42dc","Type":"ContainerStarted","Data":"211f83adf33a301736810debec1c51c3d4788d5fc3af9b17b1d68c09a2483001"} Jun 02 19:55:16.330111 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.330089 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nscs2" event={"ID":"6c422453-6b93-4de0-b000-13511f092b3f","Type":"ContainerStarted","Data":"dc2360fd19f9c6181b29ddf154d44b71d172f254b9b3e1c889d9a6cfc62f3c94"} Jun 02 19:55:16.331042 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.331016 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lz2mg" event={"ID":"c3674c37-2e21-4799-9545-d69c1985b721","Type":"ContainerStarted","Data":"4d16b215ef10667e3dc09c5064e6ab821fb7b308f879e0e625d203583c99216d"} Jun 02 19:55:16.837174 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.837080 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:16.837174 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:16.837140 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4bqvj\" (UniqueName: \"kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj\") pod \"network-check-target-62ckd\" (UID: \"ea07bb39-2258-40a3-bc5f-a05dc1e90164\") " pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:16.837397 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:16.837301 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:55:16.837397 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:16.837317 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:55:16.837397 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:16.837330 2548 projected.go:194] Error preparing data for projected volume kube-api-access-4bqvj for pod openshift-network-diagnostics/network-check-target-62ckd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:16.837397 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:16.837387 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj podName:ea07bb39-2258-40a3-bc5f-a05dc1e90164 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:18.837370392 +0000 UTC m=+6.154583629 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bqvj" (UniqueName: "kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj") pod "network-check-target-62ckd" (UID: "ea07bb39-2258-40a3-bc5f-a05dc1e90164") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:16.837849 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:16.837828 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:16.837955 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:16.837888 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs podName:c848a2cb-1edf-4aa2-b593-c0f19d847476 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:18.837871824 +0000 UTC m=+6.155085071 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs") pod "network-metrics-daemon-wwpfn" (UID: "c848a2cb-1edf-4aa2-b593-c0f19d847476") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:17.313365 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:17.313325 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:17.313853 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:17.313486 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:17.316000 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:17.315972 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:17.316137 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:17.316094 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:17.343860 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:17.343824 2548 generic.go:358] "Generic (PLEG): container finished" podID="4bcb9fdd770434633007c1d0f5480ec1" containerID="9483ae7f7bc7a560b4af804b336da2b3e377d776fbf3c9bb7b09ab32cd77f667" exitCode=0 Jun 02 19:55:17.344336 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:17.344311 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" event={"ID":"4bcb9fdd770434633007c1d0f5480ec1","Type":"ContainerDied","Data":"9483ae7f7bc7a560b4af804b336da2b3e377d776fbf3c9bb7b09ab32cd77f667"} Jun 02 19:55:17.370635 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:17.370554 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" podStartSLOduration=3.370536596 podStartE2EDuration="3.370536596s" podCreationTimestamp="2026-06-02 19:55:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:55:16.344563201 +0000 UTC m=+3.661776456" watchObservedRunningTime="2026-06-02 19:55:17.370536596 +0000 UTC m=+4.687749852" Jun 02 19:55:18.359006 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:18.358230 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" event={"ID":"4bcb9fdd770434633007c1d0f5480ec1","Type":"ContainerStarted","Data":"46c821f46394e9133274fc19ab7cf3e13d3f41757fe0c76a954c0b9984c454cb"} Jun 02 19:55:18.855423 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:18.854558 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:18.855423 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:18.854611 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4bqvj\" (UniqueName: \"kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj\") pod \"network-check-target-62ckd\" (UID: \"ea07bb39-2258-40a3-bc5f-a05dc1e90164\") " pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:18.855423 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:18.854783 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:55:18.855423 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:18.854803 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:55:18.855423 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:18.854816 2548 projected.go:194] Error preparing data for projected volume kube-api-access-4bqvj for pod openshift-network-diagnostics/network-check-target-62ckd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:18.855423 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:18.854874 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj podName:ea07bb39-2258-40a3-bc5f-a05dc1e90164 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:22.854856656 +0000 UTC m=+10.172069891 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bqvj" (UniqueName: "kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj") pod "network-check-target-62ckd" (UID: "ea07bb39-2258-40a3-bc5f-a05dc1e90164") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:18.855423 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:18.855287 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:18.855423 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:18.855338 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs podName:c848a2cb-1edf-4aa2-b593-c0f19d847476 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:22.85532306 +0000 UTC m=+10.172536297 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs") pod "network-metrics-daemon-wwpfn" (UID: "c848a2cb-1edf-4aa2-b593-c0f19d847476") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:19.315030 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:19.314283 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:19.315030 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:19.314420 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:19.315030 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:19.314798 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:19.315030 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:19.314915 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:21.314174 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.314135 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:21.314174 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.314161 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:21.314696 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:21.314282 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:21.314696 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:21.314430 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:21.528021 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.527966 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" podStartSLOduration=7.527946125 podStartE2EDuration="7.527946125s" podCreationTimestamp="2026-06-02 19:55:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:55:18.375699596 +0000 UTC m=+5.692912850" watchObservedRunningTime="2026-06-02 19:55:21.527946125 +0000 UTC m=+8.845159380" Jun 02 19:55:21.528835 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.528732 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-xzfr6"] Jun 02 19:55:21.530486 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.530453 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:21.530601 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:21.530541 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:21.578773 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.578664 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:21.578773 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.578719 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/39338125-49b0-4a5f-9591-ca49089d0da5-kubelet-config\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:21.578977 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.578795 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/39338125-49b0-4a5f-9591-ca49089d0da5-dbus\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:21.679924 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.679880 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/39338125-49b0-4a5f-9591-ca49089d0da5-kubelet-config\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:21.680107 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.680037 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/39338125-49b0-4a5f-9591-ca49089d0da5-kubelet-config\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:21.680107 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.680060 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/39338125-49b0-4a5f-9591-ca49089d0da5-dbus\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:21.680220 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.680194 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/39338125-49b0-4a5f-9591-ca49089d0da5-dbus\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:21.680220 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:21.680213 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:21.680409 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:21.680339 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:55:21.680409 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:21.680404 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret podName:39338125-49b0-4a5f-9591-ca49089d0da5 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:22.180385415 +0000 UTC m=+9.497598650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret") pod "global-pull-secret-syncer-xzfr6" (UID: "39338125-49b0-4a5f-9591-ca49089d0da5") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:55:22.185153 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:22.185107 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:22.185330 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:22.185292 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:55:22.185406 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:22.185376 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret podName:39338125-49b0-4a5f-9591-ca49089d0da5 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:23.18535493 +0000 UTC m=+10.502568171 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret") pod "global-pull-secret-syncer-xzfr6" (UID: "39338125-49b0-4a5f-9591-ca49089d0da5") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:55:22.890436 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:22.890396 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:22.890436 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:22.890451 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4bqvj\" (UniqueName: \"kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj\") pod \"network-check-target-62ckd\" (UID: \"ea07bb39-2258-40a3-bc5f-a05dc1e90164\") " pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:22.891096 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:22.890609 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:55:22.891096 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:22.890628 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:55:22.891096 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:22.890641 2548 projected.go:194] Error preparing data for projected volume kube-api-access-4bqvj for pod openshift-network-diagnostics/network-check-target-62ckd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:22.891096 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:22.890701 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj podName:ea07bb39-2258-40a3-bc5f-a05dc1e90164 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:30.890682972 +0000 UTC m=+18.207896233 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bqvj" (UniqueName: "kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj") pod "network-check-target-62ckd" (UID: "ea07bb39-2258-40a3-bc5f-a05dc1e90164") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:22.891096 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:22.890713 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:22.891096 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:22.890798 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs podName:c848a2cb-1edf-4aa2-b593-c0f19d847476 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:30.890779797 +0000 UTC m=+18.207993034 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs") pod "network-metrics-daemon-wwpfn" (UID: "c848a2cb-1edf-4aa2-b593-c0f19d847476") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:23.193570 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:23.193472 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:23.193733 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:23.193659 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:55:23.193830 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:23.193755 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret podName:39338125-49b0-4a5f-9591-ca49089d0da5 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:25.193720037 +0000 UTC m=+12.510933283 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret") pod "global-pull-secret-syncer-xzfr6" (UID: "39338125-49b0-4a5f-9591-ca49089d0da5") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:55:23.315299 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:23.314492 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:23.315299 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:23.314612 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:23.315299 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:23.315006 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:23.315299 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:23.315114 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:23.315299 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:23.315157 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:23.315299 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:23.315219 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:25.210089 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:25.210043 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:25.210546 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:25.210208 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:55:25.210546 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:25.210313 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret podName:39338125-49b0-4a5f-9591-ca49089d0da5 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:29.210258155 +0000 UTC m=+16.527471400 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret") pod "global-pull-secret-syncer-xzfr6" (UID: "39338125-49b0-4a5f-9591-ca49089d0da5") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:55:25.313562 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:25.313456 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:25.313562 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:25.313483 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:25.313562 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:25.313456 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:25.313889 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:25.313581 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:25.313889 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:25.313648 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:25.313889 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:25.313771 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:27.313162 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:27.313120 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:27.313651 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:27.313119 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:27.313651 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:27.313274 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:27.313651 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:27.313120 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:27.313651 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:27.313352 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:27.313651 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:27.313395 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:29.237579 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:29.237543 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:29.238052 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:29.237701 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:55:29.238052 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:29.237800 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret podName:39338125-49b0-4a5f-9591-ca49089d0da5 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:37.237779683 +0000 UTC m=+24.554992919 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret") pod "global-pull-secret-syncer-xzfr6" (UID: "39338125-49b0-4a5f-9591-ca49089d0da5") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:55:29.313658 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:29.313618 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:29.313658 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:29.313649 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:29.313914 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:29.313735 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:29.313914 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:29.313885 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:29.314063 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:29.314020 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:29.314209 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:29.314186 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:30.952541 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:30.952461 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:30.952541 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:30.952504 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4bqvj\" (UniqueName: \"kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj\") pod \"network-check-target-62ckd\" (UID: \"ea07bb39-2258-40a3-bc5f-a05dc1e90164\") " pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:30.953145 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:30.952622 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:30.953145 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:30.952645 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:55:30.953145 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:30.952663 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:55:30.953145 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:30.952676 2548 projected.go:194] Error preparing data for projected volume kube-api-access-4bqvj for pod openshift-network-diagnostics/network-check-target-62ckd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:30.953145 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:30.952714 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs podName:c848a2cb-1edf-4aa2-b593-c0f19d847476 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:46.952688576 +0000 UTC m=+34.269901882 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs") pod "network-metrics-daemon-wwpfn" (UID: "c848a2cb-1edf-4aa2-b593-c0f19d847476") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:30.953145 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:30.952766 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj podName:ea07bb39-2258-40a3-bc5f-a05dc1e90164 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:46.952728657 +0000 UTC m=+34.269941904 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bqvj" (UniqueName: "kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj") pod "network-check-target-62ckd" (UID: "ea07bb39-2258-40a3-bc5f-a05dc1e90164") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:31.313841 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:31.313752 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:31.314113 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:31.313752 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:31.314113 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:31.313871 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:31.314113 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:31.313951 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:31.314248 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:31.313750 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:31.314290 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:31.314250 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:33.314708 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:33.314667 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:33.315200 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:33.314789 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:33.315200 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:33.314799 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:33.315200 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:33.314898 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:33.315200 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:33.314942 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:33.315200 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:33.315030 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:35.314076 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.314031 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:35.314894 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.314031 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:35.314894 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:35.314164 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:35.314894 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.314031 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:35.314894 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:35.314304 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:35.314894 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:35.314242 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:35.391763 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.391660 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 19:55:35.392062 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.392040 2548 generic.go:358] "Generic (PLEG): container finished" podID="fcbbef18-c454-4403-861d-a1cd4cae3ce9" containerID="01fd337bafe9fff03c98fbb833b7acc35aca4d584a997b14f895abe11597f121" exitCode=1 Jun 02 19:55:35.392144 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.392117 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" event={"ID":"fcbbef18-c454-4403-861d-a1cd4cae3ce9","Type":"ContainerStarted","Data":"2cbe862e06bc1272d9c305d744791d35076b32d27942b8c3f0f56206eb902f55"} Jun 02 19:55:35.392196 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.392156 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" event={"ID":"fcbbef18-c454-4403-861d-a1cd4cae3ce9","Type":"ContainerStarted","Data":"d5caa2945c3cacacbbbf012259849ddeba3e5a27323fea471a36655bd64869ab"} Jun 02 19:55:35.392196 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.392166 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" event={"ID":"fcbbef18-c454-4403-861d-a1cd4cae3ce9","Type":"ContainerStarted","Data":"bc678946412ea76e27e98d8aa20c1adfb3ce52f7f597905777a1223436dbef65"} Jun 02 19:55:35.392196 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.392175 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" event={"ID":"fcbbef18-c454-4403-861d-a1cd4cae3ce9","Type":"ContainerStarted","Data":"19e976520a93b93e0ab08b8334aa461f2af00322dada368b43f08668a621f8ae"} Jun 02 19:55:35.392196 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.392185 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" event={"ID":"fcbbef18-c454-4403-861d-a1cd4cae3ce9","Type":"ContainerDied","Data":"01fd337bafe9fff03c98fbb833b7acc35aca4d584a997b14f895abe11597f121"} Jun 02 19:55:35.392345 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.392199 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" event={"ID":"fcbbef18-c454-4403-861d-a1cd4cae3ce9","Type":"ContainerStarted","Data":"2ad1ae6ba1cc1cc26e0d9e82cbb41b92459d607b7d542ea6851ec8c6fa449bc0"} Jun 02 19:55:35.393341 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.393320 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kcv4k" event={"ID":"861f1595-8ced-49bc-b4e3-2db727f3bb25","Type":"ContainerStarted","Data":"ad63b7e356c9f8b72a3b2e58d6404179261e78e354d3d5c339f191d7439bc799"} Jun 02 19:55:35.394483 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.394458 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2ng7z" event={"ID":"286e8706-b9e4-4ae9-b67f-3ddbcf5877a6","Type":"ContainerStarted","Data":"9eb3f7364578ec12643b0210f57df2a23ccd518e457eeadf5464ebd64adbd445"} Jun 02 19:55:35.395531 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.395513 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" event={"ID":"14272226-bd37-4c8c-94e5-221e0fd96c1c","Type":"ContainerStarted","Data":"77a7d07225bb8eefbe8ed7d836321e163bc57d3d543886e2f64de68558a65c55"} Jun 02 19:55:35.396781 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.396731 2548 generic.go:358] "Generic (PLEG): container finished" podID="4271c2ac-658b-4949-b178-ac6530158698" containerID="b62cc454cad68f105009a7b297aacbca5b591f32f4893e7f9ef33e29d0686296" exitCode=0 Jun 02 19:55:35.396871 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.396807 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5w6p" event={"ID":"4271c2ac-658b-4949-b178-ac6530158698","Type":"ContainerDied","Data":"b62cc454cad68f105009a7b297aacbca5b591f32f4893e7f9ef33e29d0686296"} Jun 02 19:55:35.398244 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.398023 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-52r6t" event={"ID":"372120bf-121b-4dff-b2b0-f89a944a42dc","Type":"ContainerStarted","Data":"898cf4c65b4dd4fe8c0c76a0037e16071c35321b5cac605e1d619aee9a82f4a1"} Jun 02 19:55:35.399376 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.399309 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nscs2" event={"ID":"6c422453-6b93-4de0-b000-13511f092b3f","Type":"ContainerStarted","Data":"e685490aa79c63ec1b1965129c46e71e87d42d6ee8ee8d26262af4d4883ef0fb"} Jun 02 19:55:35.400508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.400490 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lz2mg" event={"ID":"c3674c37-2e21-4799-9545-d69c1985b721","Type":"ContainerStarted","Data":"9fa7bfa2cfa113e13c5615933397945fd289c18aa03c3ed850222ee01d437fb4"} Jun 02 19:55:35.433944 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.433885 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kcv4k" podStartSLOduration=8.030664664 podStartE2EDuration="22.433869457s" podCreationTimestamp="2026-06-02 19:55:13 +0000 UTC" firstStartedPulling="2026-06-02 19:55:16.212935226 +0000 UTC m=+3.530148459" lastFinishedPulling="2026-06-02 19:55:30.616140017 +0000 UTC m=+17.933353252" observedRunningTime="2026-06-02 19:55:35.433764096 +0000 UTC m=+22.750977341" watchObservedRunningTime="2026-06-02 19:55:35.433869457 +0000 UTC m=+22.751082714" Jun 02 19:55:35.453140 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.453092 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-lz2mg" podStartSLOduration=4.126001111 podStartE2EDuration="22.453078376s" podCreationTimestamp="2026-06-02 19:55:13 +0000 UTC" firstStartedPulling="2026-06-02 19:55:16.229901357 +0000 UTC m=+3.547114601" lastFinishedPulling="2026-06-02 19:55:34.556978621 +0000 UTC m=+21.874191866" observedRunningTime="2026-06-02 19:55:35.45259078 +0000 UTC m=+22.769804035" watchObservedRunningTime="2026-06-02 19:55:35.453078376 +0000 UTC m=+22.770291630" Jun 02 19:55:35.478797 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.478722 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-52r6t" podStartSLOduration=4.108034484 podStartE2EDuration="22.478707942s" podCreationTimestamp="2026-06-02 19:55:13 +0000 UTC" firstStartedPulling="2026-06-02 19:55:16.226272425 +0000 UTC m=+3.543485658" lastFinishedPulling="2026-06-02 19:55:34.596945866 +0000 UTC m=+21.914159116" observedRunningTime="2026-06-02 19:55:35.478607144 +0000 UTC m=+22.795820415" watchObservedRunningTime="2026-06-02 19:55:35.478707942 +0000 UTC m=+22.795921196" Jun 02 19:55:35.545150 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.545100 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nscs2" podStartSLOduration=4.208452637 podStartE2EDuration="22.545080042s" podCreationTimestamp="2026-06-02 19:55:13 +0000 UTC" firstStartedPulling="2026-06-02 19:55:16.222401206 +0000 UTC m=+3.539614438" lastFinishedPulling="2026-06-02 19:55:34.5590286 +0000 UTC m=+21.876241843" observedRunningTime="2026-06-02 19:55:35.544795755 +0000 UTC m=+22.862009008" watchObservedRunningTime="2026-06-02 19:55:35.545080042 +0000 UTC m=+22.862293300" Jun 02 19:55:35.575295 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.575236 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2ng7z" podStartSLOduration=4.249125536 podStartE2EDuration="22.575215759s" podCreationTimestamp="2026-06-02 19:55:13 +0000 UTC" firstStartedPulling="2026-06-02 19:55:16.230921358 +0000 UTC m=+3.548134592" lastFinishedPulling="2026-06-02 19:55:34.557011572 +0000 UTC m=+21.874224815" observedRunningTime="2026-06-02 19:55:35.575147173 +0000 UTC m=+22.892360428" watchObservedRunningTime="2026-06-02 19:55:35.575215759 +0000 UTC m=+22.892429013" Jun 02 19:55:35.904019 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:35.903841 2548 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 19:55:36.179350 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:36.179185 2548 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T19:55:35.904015761Z","UUID":"198ede8c-958b-4eac-9a04-4accf2ffa762","Handler":null,"Name":"","Endpoint":""} Jun 02 19:55:36.180803 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:36.180782 2548 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 19:55:36.180803 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:36.180806 2548 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 19:55:36.404599 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:36.404561 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m4sth" event={"ID":"990f9600-65e9-4e58-80e8-862601023a1a","Type":"ContainerStarted","Data":"1a3244dbb9bdc446ed8886b5b364ae133580676d0e35bd061d34c15161f40af1"} Jun 02 19:55:36.406617 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:36.406579 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" event={"ID":"14272226-bd37-4c8c-94e5-221e0fd96c1c","Type":"ContainerStarted","Data":"467bd5eafa34dcc25e44d6f4f258222156dcd112896f632450683c45be042cd5"} Jun 02 19:55:37.298254 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:37.298220 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:37.298434 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:37.298412 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:55:37.298509 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:37.298498 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret podName:39338125-49b0-4a5f-9591-ca49089d0da5 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:53.298477045 +0000 UTC m=+40.615690298 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret") pod "global-pull-secret-syncer-xzfr6" (UID: "39338125-49b0-4a5f-9591-ca49089d0da5") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:55:37.313905 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:37.313870 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:37.314052 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:37.313870 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:37.314052 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:37.313976 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:37.314172 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:37.313874 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:37.314172 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:37.314075 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:37.314172 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:37.314155 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:37.411459 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:37.411424 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 19:55:37.412242 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:37.411888 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" event={"ID":"fcbbef18-c454-4403-861d-a1cd4cae3ce9","Type":"ContainerStarted","Data":"d701c5543127dfd63c597597c18fc5f417303cbed7e80cdceb3ef930de963765"} Jun 02 19:55:37.414039 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:37.414012 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" event={"ID":"14272226-bd37-4c8c-94e5-221e0fd96c1c","Type":"ContainerStarted","Data":"bc9c249d1587c0467f7050d759adaec8f237fd397a0b8e1a923e4c706141ef37"} Jun 02 19:55:37.443067 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:37.442965 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m4sth" podStartSLOduration=10.044406881 podStartE2EDuration="24.442947318s" podCreationTimestamp="2026-06-02 19:55:13 +0000 UTC" firstStartedPulling="2026-06-02 19:55:16.217557059 +0000 UTC m=+3.534770291" lastFinishedPulling="2026-06-02 19:55:30.616097489 +0000 UTC m=+17.933310728" observedRunningTime="2026-06-02 19:55:36.43367283 +0000 UTC m=+23.750886083" watchObservedRunningTime="2026-06-02 19:55:37.442947318 +0000 UTC m=+24.760160573" Jun 02 19:55:38.886533 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:38.886491 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2ng7z" Jun 02 19:55:38.887340 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:38.887320 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2ng7z" Jun 02 19:55:38.914374 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:38.914324 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7hnl" podStartSLOduration=5.227244383 podStartE2EDuration="25.914307389s" podCreationTimestamp="2026-06-02 19:55:13 +0000 UTC" firstStartedPulling="2026-06-02 19:55:16.228478135 +0000 UTC m=+3.545691379" lastFinishedPulling="2026-06-02 19:55:36.915541139 +0000 UTC m=+24.232754385" observedRunningTime="2026-06-02 19:55:37.443512069 +0000 UTC m=+24.760725324" watchObservedRunningTime="2026-06-02 19:55:38.914307389 +0000 UTC m=+26.231520646" Jun 02 19:55:39.313780 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:39.313717 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:39.313983 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:39.313794 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:39.313983 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:39.313922 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:39.314091 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:39.314015 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:39.314132 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:39.314098 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:39.314195 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:39.314176 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:39.417841 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:39.417800 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2ng7z" Jun 02 19:55:39.418584 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:39.418561 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2ng7z" Jun 02 19:55:41.314198 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:41.313957 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:41.314808 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:41.313965 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:41.314808 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:41.314230 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:41.314808 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:41.313968 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:41.314808 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:41.314293 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:41.314808 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:41.314363 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:41.424146 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:41.424113 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 19:55:41.424508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:41.424486 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" event={"ID":"fcbbef18-c454-4403-861d-a1cd4cae3ce9","Type":"ContainerStarted","Data":"f827ed8c583fca05bccd4bb1e0d75208e5f5038a61a32cc627d93c19031f6ae7"} Jun 02 19:55:41.424870 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:41.424851 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:41.425000 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:41.424985 2548 scope.go:117] "RemoveContainer" containerID="01fd337bafe9fff03c98fbb833b7acc35aca4d584a997b14f895abe11597f121" Jun 02 19:55:41.426279 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:41.426255 2548 generic.go:358] "Generic (PLEG): container finished" podID="4271c2ac-658b-4949-b178-ac6530158698" containerID="840331fa9b81678b451a4bb3fe53b34a887a60e29ab11ffa377d3689f6f2924b" exitCode=0 Jun 02 19:55:41.426369 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:41.426339 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5w6p" event={"ID":"4271c2ac-658b-4949-b178-ac6530158698","Type":"ContainerDied","Data":"840331fa9b81678b451a4bb3fe53b34a887a60e29ab11ffa377d3689f6f2924b"} Jun 02 19:55:41.442678 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:41.442623 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:42.431101 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:42.431065 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 19:55:42.431521 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:42.431427 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" event={"ID":"fcbbef18-c454-4403-861d-a1cd4cae3ce9","Type":"ContainerStarted","Data":"4066a1aae70aa0ed58e302b6e372427918c269cfa7e67783a8b4c17026742956"} Jun 02 19:55:42.431771 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:42.431755 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:42.431863 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:42.431778 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:42.447253 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:42.447220 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:55:42.464482 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:42.464428 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" podStartSLOduration=11.110882412 podStartE2EDuration="29.464410964s" podCreationTimestamp="2026-06-02 19:55:13 +0000 UTC" firstStartedPulling="2026-06-02 19:55:16.22350746 +0000 UTC m=+3.540720692" lastFinishedPulling="2026-06-02 19:55:34.577036009 +0000 UTC m=+21.894249244" observedRunningTime="2026-06-02 19:55:42.463844501 +0000 UTC m=+29.781057770" watchObservedRunningTime="2026-06-02 19:55:42.464410964 +0000 UTC m=+29.781624216" Jun 02 19:55:42.741863 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:42.741556 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-62ckd"] Jun 02 19:55:42.741984 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:42.741945 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:42.742072 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:42.742053 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:42.745205 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:42.745175 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xzfr6"] Jun 02 19:55:42.745338 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:42.745292 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:42.745414 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:42.745391 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:42.745767 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:42.745730 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wwpfn"] Jun 02 19:55:42.745881 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:42.745846 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:42.745966 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:42.745942 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:43.435266 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:43.435235 2548 generic.go:358] "Generic (PLEG): container finished" podID="4271c2ac-658b-4949-b178-ac6530158698" containerID="664d88a8dbd29d2ff45a350044c5877ac997ed54537d4fff5d42191b5cf41bc7" exitCode=0 Jun 02 19:55:43.435727 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:43.435294 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5w6p" event={"ID":"4271c2ac-658b-4949-b178-ac6530158698","Type":"ContainerDied","Data":"664d88a8dbd29d2ff45a350044c5877ac997ed54537d4fff5d42191b5cf41bc7"} Jun 02 19:55:44.314086 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:44.314055 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:44.314219 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:44.314124 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:44.314219 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:44.314206 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:44.314294 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:44.314216 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:44.314294 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:44.314278 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:44.314371 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:44.314342 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:44.439249 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:44.439219 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5w6p" event={"ID":"4271c2ac-658b-4949-b178-ac6530158698","Type":"ContainerStarted","Data":"35506d846a366814f3a46f584e08e8d8eca9498c405ed926312380aed80d7467"} Jun 02 19:55:45.442630 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:45.442598 2548 generic.go:358] "Generic (PLEG): container finished" podID="4271c2ac-658b-4949-b178-ac6530158698" containerID="35506d846a366814f3a46f584e08e8d8eca9498c405ed926312380aed80d7467" exitCode=0 Jun 02 19:55:45.443170 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:45.442653 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5w6p" event={"ID":"4271c2ac-658b-4949-b178-ac6530158698","Type":"ContainerDied","Data":"35506d846a366814f3a46f584e08e8d8eca9498c405ed926312380aed80d7467"} Jun 02 19:55:46.313761 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:46.313701 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:46.314106 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:46.313701 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:46.314106 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:46.313701 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:46.314106 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:46.313852 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62ckd" podUID="ea07bb39-2258-40a3-bc5f-a05dc1e90164" Jun 02 19:55:46.314106 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:46.313901 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzfr6" podUID="39338125-49b0-4a5f-9591-ca49089d0da5" Jun 02 19:55:46.314106 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:46.313963 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:55:46.974128 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:46.974088 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:46.974128 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:46.974125 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4bqvj\" (UniqueName: \"kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj\") pod \"network-check-target-62ckd\" (UID: \"ea07bb39-2258-40a3-bc5f-a05dc1e90164\") " pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:46.974715 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:46.974223 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:55:46.974715 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:46.974222 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:46.974715 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:46.974235 2548 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:55:46.974715 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:46.974247 2548 projected.go:194] Error preparing data for projected volume kube-api-access-4bqvj for pod openshift-network-diagnostics/network-check-target-62ckd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:46.974715 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:46.974283 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj podName:ea07bb39-2258-40a3-bc5f-a05dc1e90164 nodeName:}" failed. No retries permitted until 2026-06-02 19:56:18.974272353 +0000 UTC m=+66.291485584 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bqvj" (UniqueName: "kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj") pod "network-check-target-62ckd" (UID: "ea07bb39-2258-40a3-bc5f-a05dc1e90164") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:55:46.974715 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:46.974296 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs podName:c848a2cb-1edf-4aa2-b593-c0f19d847476 nodeName:}" failed. No retries permitted until 2026-06-02 19:56:18.974289903 +0000 UTC m=+66.291503135 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs") pod "network-metrics-daemon-wwpfn" (UID: "c848a2cb-1edf-4aa2-b593-c0f19d847476") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:55:47.471275 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.471249 2548 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeReady" Jun 02 19:55:47.471441 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.471379 2548 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 19:55:47.516611 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.516578 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6884588f7d-rvb4x"] Jun 02 19:55:47.541598 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.541564 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.542026 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.541835 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6884588f7d-rvb4x"] Jun 02 19:55:47.545037 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.545011 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 19:55:47.545162 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.545058 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 19:55:47.545578 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.545558 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 19:55:47.545671 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.545584 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vtp5s\"" Jun 02 19:55:47.551428 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.551407 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rmbfk"] Jun 02 19:55:47.555104 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.555085 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 19:55:47.568779 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.568733 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jjfnh"] Jun 02 19:55:47.584995 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.584956 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jjfnh"] Jun 02 19:55:47.584995 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.584988 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rmbfk"] Jun 02 19:55:47.585238 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.585106 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:55:47.585238 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.585112 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:47.588433 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.588410 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 19:55:47.588602 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.588585 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 19:55:47.588684 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.588476 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 19:55:47.588763 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.588525 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-l7ztv\"" Jun 02 19:55:47.588763 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.588585 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 19:55:47.588866 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.588423 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mg65m\"" Jun 02 19:55:47.589048 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.589029 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 19:55:47.679658 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.679614 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-certificates\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.679658 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.679654 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d58c985a-62d6-4fa6-85f2-a79d3cebac80-installation-pull-secrets\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.679909 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.679689 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:47.679909 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.679823 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmm9v\" (UniqueName: \"kubernetes.io/projected/7952dddc-f51b-4a0c-8a23-feecfec0457a-kube-api-access-cmm9v\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:47.679909 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.679867 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7952dddc-f51b-4a0c-8a23-feecfec0457a-tmp-dir\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:47.680050 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.679919 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.680050 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.679946 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-bound-sa-token\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.680050 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.679973 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d58c985a-62d6-4fa6-85f2-a79d3cebac80-image-registry-private-configuration\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.680050 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.680000 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d58c985a-62d6-4fa6-85f2-a79d3cebac80-trusted-ca\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.680050 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.680021 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:55:47.680050 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.680046 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhjgn\" (UniqueName: \"kubernetes.io/projected/3bade01b-4276-40ba-b81a-d607f88064e2-kube-api-access-jhjgn\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:55:47.680333 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.680080 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d58c985a-62d6-4fa6-85f2-a79d3cebac80-ca-trust-extracted\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.680333 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.680099 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45h55\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-kube-api-access-45h55\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.680333 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.680148 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7952dddc-f51b-4a0c-8a23-feecfec0457a-config-volume\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:47.780530 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.780443 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d58c985a-62d6-4fa6-85f2-a79d3cebac80-trusted-ca\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.780530 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.780476 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:55:47.780530 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.780497 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jhjgn\" (UniqueName: \"kubernetes.io/projected/3bade01b-4276-40ba-b81a-d607f88064e2-kube-api-access-jhjgn\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:55:47.780820 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:47.780581 2548 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:55:47.780820 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:47.780643 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert podName:3bade01b-4276-40ba-b81a-d607f88064e2 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:48.280623001 +0000 UTC m=+35.597836233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert") pod "ingress-canary-jjfnh" (UID: "3bade01b-4276-40ba-b81a-d607f88064e2") : secret "canary-serving-cert" not found Jun 02 19:55:47.780820 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.780669 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d58c985a-62d6-4fa6-85f2-a79d3cebac80-ca-trust-extracted\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.780820 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.780692 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45h55\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-kube-api-access-45h55\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.780820 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.780712 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7952dddc-f51b-4a0c-8a23-feecfec0457a-config-volume\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:47.780820 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.780730 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-certificates\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.780820 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.780776 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d58c985a-62d6-4fa6-85f2-a79d3cebac80-installation-pull-secrets\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.780820 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.780812 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:47.781210 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.780862 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cmm9v\" (UniqueName: \"kubernetes.io/projected/7952dddc-f51b-4a0c-8a23-feecfec0457a-kube-api-access-cmm9v\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:47.781210 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.780891 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7952dddc-f51b-4a0c-8a23-feecfec0457a-tmp-dir\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:47.781210 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:47.780968 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:55:47.781210 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:47.781018 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls podName:7952dddc-f51b-4a0c-8a23-feecfec0457a nodeName:}" failed. No retries permitted until 2026-06-02 19:55:48.281005357 +0000 UTC m=+35.598218589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls") pod "dns-default-rmbfk" (UID: "7952dddc-f51b-4a0c-8a23-feecfec0457a") : secret "dns-default-metrics-tls" not found Jun 02 19:55:47.781210 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.781014 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.781210 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.781046 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-bound-sa-token\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.781210 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.781069 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d58c985a-62d6-4fa6-85f2-a79d3cebac80-image-registry-private-configuration\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.781210 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:47.781073 2548 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 19:55:47.781210 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:47.781090 2548 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884588f7d-rvb4x: secret "image-registry-tls" not found Jun 02 19:55:47.781210 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:47.781145 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls podName:d58c985a-62d6-4fa6-85f2-a79d3cebac80 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:48.281128863 +0000 UTC m=+35.598342102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls") pod "image-registry-6884588f7d-rvb4x" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80") : secret "image-registry-tls" not found Jun 02 19:55:47.781210 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.781194 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7952dddc-f51b-4a0c-8a23-feecfec0457a-tmp-dir\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:47.781590 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.781309 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d58c985a-62d6-4fa6-85f2-a79d3cebac80-ca-trust-extracted\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.781590 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.781506 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d58c985a-62d6-4fa6-85f2-a79d3cebac80-trusted-ca\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.781590 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.781506 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7952dddc-f51b-4a0c-8a23-feecfec0457a-config-volume\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:47.781679 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.781645 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-certificates\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.785548 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.785374 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d58c985a-62d6-4fa6-85f2-a79d3cebac80-installation-pull-secrets\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.785680 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.785381 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d58c985a-62d6-4fa6-85f2-a79d3cebac80-image-registry-private-configuration\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.790901 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.790872 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmm9v\" (UniqueName: \"kubernetes.io/projected/7952dddc-f51b-4a0c-8a23-feecfec0457a-kube-api-access-cmm9v\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:47.791198 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.791170 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-bound-sa-token\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:47.791447 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.791428 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhjgn\" (UniqueName: \"kubernetes.io/projected/3bade01b-4276-40ba-b81a-d607f88064e2-kube-api-access-jhjgn\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:55:47.791592 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:47.791555 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45h55\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-kube-api-access-45h55\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:48.285560 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:48.285522 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:48.285560 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:48.285560 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:55:48.286090 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:48.285606 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:48.286090 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:48.285677 2548 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 19:55:48.286090 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:48.285699 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:55:48.286090 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:48.285700 2548 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884588f7d-rvb4x: secret "image-registry-tls" not found Jun 02 19:55:48.286090 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:48.285767 2548 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:55:48.286090 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:48.285772 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls podName:7952dddc-f51b-4a0c-8a23-feecfec0457a nodeName:}" failed. No retries permitted until 2026-06-02 19:55:49.285736237 +0000 UTC m=+36.602949468 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls") pod "dns-default-rmbfk" (UID: "7952dddc-f51b-4a0c-8a23-feecfec0457a") : secret "dns-default-metrics-tls" not found Jun 02 19:55:48.286090 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:48.285858 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls podName:d58c985a-62d6-4fa6-85f2-a79d3cebac80 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:49.285836816 +0000 UTC m=+36.603050062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls") pod "image-registry-6884588f7d-rvb4x" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80") : secret "image-registry-tls" not found Jun 02 19:55:48.286090 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:48.285873 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert podName:3bade01b-4276-40ba-b81a-d607f88064e2 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:49.28586489 +0000 UTC m=+36.603078123 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert") pod "ingress-canary-jjfnh" (UID: "3bade01b-4276-40ba-b81a-d607f88064e2") : secret "canary-serving-cert" not found Jun 02 19:55:48.314150 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:48.314107 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:48.314336 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:48.314220 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:55:48.314336 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:48.314309 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:55:48.318222 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:48.318202 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 19:55:48.318369 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:48.318209 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 19:55:48.319437 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:48.319416 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r9fgf\"" Jun 02 19:55:48.319575 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:48.319415 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dzx54\"" Jun 02 19:55:48.319823 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:48.319803 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 19:55:48.319935 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:48.319855 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 19:55:49.293785 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:49.293732 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:55:49.294156 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:49.293852 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:49.294156 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:49.293890 2548 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:55:49.294156 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:49.293901 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:49.294156 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:49.293954 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert podName:3bade01b-4276-40ba-b81a-d607f88064e2 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:51.293938774 +0000 UTC m=+38.611152027 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert") pod "ingress-canary-jjfnh" (UID: "3bade01b-4276-40ba-b81a-d607f88064e2") : secret "canary-serving-cert" not found Jun 02 19:55:49.294156 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:49.293984 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:55:49.294156 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:49.294034 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls podName:7952dddc-f51b-4a0c-8a23-feecfec0457a nodeName:}" failed. No retries permitted until 2026-06-02 19:55:51.294019207 +0000 UTC m=+38.611232439 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls") pod "dns-default-rmbfk" (UID: "7952dddc-f51b-4a0c-8a23-feecfec0457a") : secret "dns-default-metrics-tls" not found Jun 02 19:55:49.294156 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:49.293985 2548 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 19:55:49.294156 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:49.294053 2548 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884588f7d-rvb4x: secret "image-registry-tls" not found Jun 02 19:55:49.294156 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:49.294078 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls podName:d58c985a-62d6-4fa6-85f2-a79d3cebac80 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:51.29407198 +0000 UTC m=+38.611285211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls") pod "image-registry-6884588f7d-rvb4x" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80") : secret "image-registry-tls" not found Jun 02 19:55:51.310675 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:51.310543 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:51.310675 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:51.310605 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:55:51.310675 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:51.310677 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:51.311247 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:51.310692 2548 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 19:55:51.311247 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:51.310714 2548 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884588f7d-rvb4x: secret "image-registry-tls" not found Jun 02 19:55:51.311247 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:51.310777 2548 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:55:51.311247 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:51.310797 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls podName:d58c985a-62d6-4fa6-85f2-a79d3cebac80 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:55.310770933 +0000 UTC m=+42.627984169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls") pod "image-registry-6884588f7d-rvb4x" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80") : secret "image-registry-tls" not found Jun 02 19:55:51.311247 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:51.310797 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:55:51.311247 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:51.310849 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert podName:3bade01b-4276-40ba-b81a-d607f88064e2 nodeName:}" failed. No retries permitted until 2026-06-02 19:55:55.31082289 +0000 UTC m=+42.628036136 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert") pod "ingress-canary-jjfnh" (UID: "3bade01b-4276-40ba-b81a-d607f88064e2") : secret "canary-serving-cert" not found Jun 02 19:55:51.311247 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:51.310873 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls podName:7952dddc-f51b-4a0c-8a23-feecfec0457a nodeName:}" failed. No retries permitted until 2026-06-02 19:55:55.310861382 +0000 UTC m=+42.628074618 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls") pod "dns-default-rmbfk" (UID: "7952dddc-f51b-4a0c-8a23-feecfec0457a") : secret "dns-default-metrics-tls" not found Jun 02 19:55:53.326559 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:53.326520 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:53.338471 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:53.338443 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39338125-49b0-4a5f-9591-ca49089d0da5-original-pull-secret\") pod \"global-pull-secret-syncer-xzfr6\" (UID: \"39338125-49b0-4a5f-9591-ca49089d0da5\") " pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:53.425109 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:53.425068 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzfr6" Jun 02 19:55:54.691558 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:54.691529 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xzfr6"] Jun 02 19:55:54.695633 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:55:54.695599 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39338125_49b0_4a5f_9591_ca49089d0da5.slice/crio-268cd95a02c303df150d3a56c303218c63bf475d732e9a62bab80cb77df387d2 WatchSource:0}: Error finding container 268cd95a02c303df150d3a56c303218c63bf475d732e9a62bab80cb77df387d2: Status 404 returned error can't find the container with id 268cd95a02c303df150d3a56c303218c63bf475d732e9a62bab80cb77df387d2 Jun 02 19:55:55.339651 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:55.339611 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:55:55.339869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:55.339686 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:55:55.339869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:55.339714 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:55:55.339869 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:55.339801 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:55:55.339869 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:55.339844 2548 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 19:55:55.339869 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:55.339866 2548 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884588f7d-rvb4x: secret "image-registry-tls" not found Jun 02 19:55:55.340134 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:55.339876 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls podName:7952dddc-f51b-4a0c-8a23-feecfec0457a nodeName:}" failed. No retries permitted until 2026-06-02 19:56:03.339855936 +0000 UTC m=+50.657069173 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls") pod "dns-default-rmbfk" (UID: "7952dddc-f51b-4a0c-8a23-feecfec0457a") : secret "dns-default-metrics-tls" not found Jun 02 19:55:55.340134 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:55.339902 2548 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:55:55.340134 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:55.339922 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls podName:d58c985a-62d6-4fa6-85f2-a79d3cebac80 nodeName:}" failed. No retries permitted until 2026-06-02 19:56:03.339903126 +0000 UTC m=+50.657116367 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls") pod "image-registry-6884588f7d-rvb4x" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80") : secret "image-registry-tls" not found Jun 02 19:55:55.340134 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:55:55.339943 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert podName:3bade01b-4276-40ba-b81a-d607f88064e2 nodeName:}" failed. No retries permitted until 2026-06-02 19:56:03.339930576 +0000 UTC m=+50.657143808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert") pod "ingress-canary-jjfnh" (UID: "3bade01b-4276-40ba-b81a-d607f88064e2") : secret "canary-serving-cert" not found Jun 02 19:55:55.466537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:55.466488 2548 generic.go:358] "Generic (PLEG): container finished" podID="4271c2ac-658b-4949-b178-ac6530158698" containerID="3a83b1ed3eb577fd77f4efba90288b51693e6cbdfb2dbb3ceec4512484fc0b0a" exitCode=0 Jun 02 19:55:55.466708 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:55.466579 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5w6p" event={"ID":"4271c2ac-658b-4949-b178-ac6530158698","Type":"ContainerDied","Data":"3a83b1ed3eb577fd77f4efba90288b51693e6cbdfb2dbb3ceec4512484fc0b0a"} Jun 02 19:55:55.467838 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:55.467797 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xzfr6" event={"ID":"39338125-49b0-4a5f-9591-ca49089d0da5","Type":"ContainerStarted","Data":"268cd95a02c303df150d3a56c303218c63bf475d732e9a62bab80cb77df387d2"} Jun 02 19:55:56.473484 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:56.473448 2548 generic.go:358] "Generic (PLEG): container finished" podID="4271c2ac-658b-4949-b178-ac6530158698" containerID="893811a16d362d308d91fb524856b61521351f59044ca3bd3e2cd08fe6bdb169" exitCode=0 Jun 02 19:55:56.473938 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:56.473516 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5w6p" event={"ID":"4271c2ac-658b-4949-b178-ac6530158698","Type":"ContainerDied","Data":"893811a16d362d308d91fb524856b61521351f59044ca3bd3e2cd08fe6bdb169"} Jun 02 19:55:57.479174 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:57.479140 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5w6p" event={"ID":"4271c2ac-658b-4949-b178-ac6530158698","Type":"ContainerStarted","Data":"deb11b622d5fc05784be92bd0017c167521c06d6319984af17e45570e18fb763"} Jun 02 19:55:57.504032 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:57.503967 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j5w6p" podStartSLOduration=6.13490032 podStartE2EDuration="44.503949716s" podCreationTimestamp="2026-06-02 19:55:13 +0000 UTC" firstStartedPulling="2026-06-02 19:55:16.22707809 +0000 UTC m=+3.544291322" lastFinishedPulling="2026-06-02 19:55:54.596127481 +0000 UTC m=+41.913340718" observedRunningTime="2026-06-02 19:55:57.502867486 +0000 UTC m=+44.820080740" watchObservedRunningTime="2026-06-02 19:55:57.503949716 +0000 UTC m=+44.821162973" Jun 02 19:55:58.482720 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:58.482683 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xzfr6" event={"ID":"39338125-49b0-4a5f-9591-ca49089d0da5","Type":"ContainerStarted","Data":"20df45f5b750151d6ace58b44ec0aff02ecd02f39be4c60fe67837af01579a17"} Jun 02 19:55:58.501075 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:55:58.500974 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xzfr6" podStartSLOduration=33.952417048 podStartE2EDuration="37.500959129s" podCreationTimestamp="2026-06-02 19:55:21 +0000 UTC" firstStartedPulling="2026-06-02 19:55:54.697883522 +0000 UTC m=+42.015096769" lastFinishedPulling="2026-06-02 19:55:58.246425604 +0000 UTC m=+45.563638850" observedRunningTime="2026-06-02 19:55:58.499719578 +0000 UTC m=+45.816932832" watchObservedRunningTime="2026-06-02 19:55:58.500959129 +0000 UTC m=+45.818172383" Jun 02 19:56:03.403089 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:03.403044 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:56:03.403089 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:03.403088 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:56:03.403639 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:03.403178 2548 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:56:03.403639 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:03.403184 2548 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 19:56:03.403639 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:03.403202 2548 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884588f7d-rvb4x: secret "image-registry-tls" not found Jun 02 19:56:03.403639 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:03.403235 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert podName:3bade01b-4276-40ba-b81a-d607f88064e2 nodeName:}" failed. No retries permitted until 2026-06-02 19:56:19.403220018 +0000 UTC m=+66.720433250 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert") pod "ingress-canary-jjfnh" (UID: "3bade01b-4276-40ba-b81a-d607f88064e2") : secret "canary-serving-cert" not found Jun 02 19:56:03.403639 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:03.403301 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls podName:d58c985a-62d6-4fa6-85f2-a79d3cebac80 nodeName:}" failed. No retries permitted until 2026-06-02 19:56:19.403285176 +0000 UTC m=+66.720498408 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls") pod "image-registry-6884588f7d-rvb4x" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80") : secret "image-registry-tls" not found Jun 02 19:56:03.403639 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:03.403349 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:56:03.403639 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:03.403431 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:56:03.403639 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:03.403472 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls podName:7952dddc-f51b-4a0c-8a23-feecfec0457a nodeName:}" failed. No retries permitted until 2026-06-02 19:56:19.403464973 +0000 UTC m=+66.720678204 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls") pod "dns-default-rmbfk" (UID: "7952dddc-f51b-4a0c-8a23-feecfec0457a") : secret "dns-default-metrics-tls" not found Jun 02 19:56:14.455858 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:14.455828 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-np2n4" Jun 02 19:56:19.016416 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.016365 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:56:19.016416 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.016414 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4bqvj\" (UniqueName: \"kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj\") pod \"network-check-target-62ckd\" (UID: \"ea07bb39-2258-40a3-bc5f-a05dc1e90164\") " pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:56:19.019111 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.019090 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 19:56:19.019272 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.019254 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 19:56:19.026820 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:19.026800 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 19:56:19.026884 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:19.026874 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs podName:c848a2cb-1edf-4aa2-b593-c0f19d847476 nodeName:}" failed. No retries permitted until 2026-06-02 19:57:23.026857195 +0000 UTC m=+130.344070432 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs") pod "network-metrics-daemon-wwpfn" (UID: "c848a2cb-1edf-4aa2-b593-c0f19d847476") : secret "metrics-daemon-secret" not found Jun 02 19:56:19.029541 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.029522 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 19:56:19.039785 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.039733 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bqvj\" (UniqueName: \"kubernetes.io/projected/ea07bb39-2258-40a3-bc5f-a05dc1e90164-kube-api-access-4bqvj\") pod \"network-check-target-62ckd\" (UID: \"ea07bb39-2258-40a3-bc5f-a05dc1e90164\") " pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:56:19.234241 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.234209 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dzx54\"" Jun 02 19:56:19.241113 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.241085 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:56:19.379530 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.379494 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-62ckd"] Jun 02 19:56:19.383085 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:56:19.383053 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea07bb39_2258_40a3_bc5f_a05dc1e90164.slice/crio-2acab426e95c307df1ce6a3846acae2c113143f34d4e2eec5a899ec39a5f1447 WatchSource:0}: Error finding container 2acab426e95c307df1ce6a3846acae2c113143f34d4e2eec5a899ec39a5f1447: Status 404 returned error can't find the container with id 2acab426e95c307df1ce6a3846acae2c113143f34d4e2eec5a899ec39a5f1447 Jun 02 19:56:19.418821 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.418785 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:56:19.419005 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.418848 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:56:19.419005 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.418872 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:56:19.419005 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:19.418941 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:56:19.419005 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:19.418970 2548 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:56:19.419005 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:19.418993 2548 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 19:56:19.419188 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:19.419008 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls podName:7952dddc-f51b-4a0c-8a23-feecfec0457a nodeName:}" failed. No retries permitted until 2026-06-02 19:56:51.418990756 +0000 UTC m=+98.736203988 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls") pod "dns-default-rmbfk" (UID: "7952dddc-f51b-4a0c-8a23-feecfec0457a") : secret "dns-default-metrics-tls" not found Jun 02 19:56:19.419188 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:19.419009 2548 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884588f7d-rvb4x: secret "image-registry-tls" not found Jun 02 19:56:19.419188 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:19.419022 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert podName:3bade01b-4276-40ba-b81a-d607f88064e2 nodeName:}" failed. No retries permitted until 2026-06-02 19:56:51.419016194 +0000 UTC m=+98.736229426 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert") pod "ingress-canary-jjfnh" (UID: "3bade01b-4276-40ba-b81a-d607f88064e2") : secret "canary-serving-cert" not found Jun 02 19:56:19.419188 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:19.419058 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls podName:d58c985a-62d6-4fa6-85f2-a79d3cebac80 nodeName:}" failed. No retries permitted until 2026-06-02 19:56:51.419044925 +0000 UTC m=+98.736258157 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls") pod "image-registry-6884588f7d-rvb4x" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80") : secret "image-registry-tls" not found Jun 02 19:56:19.525856 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:19.525815 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-62ckd" event={"ID":"ea07bb39-2258-40a3-bc5f-a05dc1e90164","Type":"ContainerStarted","Data":"2acab426e95c307df1ce6a3846acae2c113143f34d4e2eec5a899ec39a5f1447"} Jun 02 19:56:23.534877 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:23.534839 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-62ckd" event={"ID":"ea07bb39-2258-40a3-bc5f-a05dc1e90164","Type":"ContainerStarted","Data":"6abe14dd7058a8a4f346221e3c4d699d0bd4ac4ac2e0e930c6fbd9237de8428e"} Jun 02 19:56:23.535249 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:23.534968 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:56:23.568914 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:23.568864 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-62ckd" podStartSLOduration=67.319095651 podStartE2EDuration="1m10.568848208s" podCreationTimestamp="2026-06-02 19:55:13 +0000 UTC" firstStartedPulling="2026-06-02 19:56:19.3856457 +0000 UTC m=+66.702858932" lastFinishedPulling="2026-06-02 19:56:22.635398249 +0000 UTC m=+69.952611489" observedRunningTime="2026-06-02 19:56:23.568696856 +0000 UTC m=+70.885910110" watchObservedRunningTime="2026-06-02 19:56:23.568848208 +0000 UTC m=+70.886061458" Jun 02 19:56:51.448954 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:51.448916 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:56:51.448954 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:51.448961 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:56:51.449484 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:51.449059 2548 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 19:56:51.449484 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:51.449079 2548 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884588f7d-rvb4x: secret "image-registry-tls" not found Jun 02 19:56:51.449484 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:51.449110 2548 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:56:51.449484 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:51.449130 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls podName:d58c985a-62d6-4fa6-85f2-a79d3cebac80 nodeName:}" failed. No retries permitted until 2026-06-02 19:57:55.449114067 +0000 UTC m=+162.766327298 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls") pod "image-registry-6884588f7d-rvb4x" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80") : secret "image-registry-tls" not found Jun 02 19:56:51.449484 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:51.449158 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert podName:3bade01b-4276-40ba-b81a-d607f88064e2 nodeName:}" failed. No retries permitted until 2026-06-02 19:57:55.449142748 +0000 UTC m=+162.766355997 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert") pod "ingress-canary-jjfnh" (UID: "3bade01b-4276-40ba-b81a-d607f88064e2") : secret "canary-serving-cert" not found Jun 02 19:56:51.449484 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:51.449209 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:56:51.449484 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:51.449295 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:56:51.449484 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:56:51.449328 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls podName:7952dddc-f51b-4a0c-8a23-feecfec0457a nodeName:}" failed. No retries permitted until 2026-06-02 19:57:55.44931734 +0000 UTC m=+162.766530575 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls") pod "dns-default-rmbfk" (UID: "7952dddc-f51b-4a0c-8a23-feecfec0457a") : secret "dns-default-metrics-tls" not found Jun 02 19:56:54.539596 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:56:54.539564 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-62ckd" Jun 02 19:57:03.956683 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:03.956645 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd"] Jun 02 19:57:03.958469 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:03.958453 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:03.962067 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:03.962038 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 02 19:57:03.964517 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:03.964460 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 02 19:57:03.964602 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:03.964513 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 19:57:03.964602 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:03.964538 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-fbhzp\"" Jun 02 19:57:03.964846 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:03.964680 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 19:57:03.980331 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:03.980297 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd"] Jun 02 19:57:04.044892 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.044852 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvbd9\" (UniqueName: \"kubernetes.io/projected/cc66e3d3-9282-4836-af26-6438bb0179dc-kube-api-access-wvbd9\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:04.044892 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.044893 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:04.045122 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.044999 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cc66e3d3-9282-4836-af26-6438bb0179dc-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:04.055688 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.055659 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-788bf58df5-kgq9t"] Jun 02 19:57:04.057409 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.057393 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.059785 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.059767 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 02 19:57:04.060413 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.060394 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 02 19:57:04.060497 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.060423 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-tx4fs\"" Jun 02 19:57:04.060685 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.060664 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 19:57:04.060777 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.060666 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 19:57:04.065486 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.065461 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 02 19:57:04.075615 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.075580 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-788bf58df5-kgq9t"] Jun 02 19:57:04.145919 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.145873 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cc66e3d3-9282-4836-af26-6438bb0179dc-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:04.145919 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.145919 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1493c956-abd3-4deb-be9c-9ac0eb2eb886-tmp\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.146180 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.145938 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1493c956-abd3-4deb-be9c-9ac0eb2eb886-snapshots\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.146180 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.145957 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1493c956-abd3-4deb-be9c-9ac0eb2eb886-service-ca-bundle\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.146180 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.146033 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzsqs\" (UniqueName: \"kubernetes.io/projected/1493c956-abd3-4deb-be9c-9ac0eb2eb886-kube-api-access-fzsqs\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.146180 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.146086 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1493c956-abd3-4deb-be9c-9ac0eb2eb886-serving-cert\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.146180 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.146164 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wvbd9\" (UniqueName: \"kubernetes.io/projected/cc66e3d3-9282-4836-af26-6438bb0179dc-kube-api-access-wvbd9\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:04.146346 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.146193 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:04.146346 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.146227 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1493c956-abd3-4deb-be9c-9ac0eb2eb886-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.146346 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:04.146318 2548 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 19:57:04.146456 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:04.146369 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls podName:cc66e3d3-9282-4836-af26-6438bb0179dc nodeName:}" failed. No retries permitted until 2026-06-02 19:57:04.646353748 +0000 UTC m=+111.963566980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-z57gd" (UID: "cc66e3d3-9282-4836-af26-6438bb0179dc") : secret "cluster-monitoring-operator-tls" not found Jun 02 19:57:04.146686 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.146669 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cc66e3d3-9282-4836-af26-6438bb0179dc-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:04.156095 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.156072 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvbd9\" (UniqueName: \"kubernetes.io/projected/cc66e3d3-9282-4836-af26-6438bb0179dc-kube-api-access-wvbd9\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:04.247618 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.247521 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1493c956-abd3-4deb-be9c-9ac0eb2eb886-tmp\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.247618 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.247563 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1493c956-abd3-4deb-be9c-9ac0eb2eb886-snapshots\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.247852 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.247677 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1493c956-abd3-4deb-be9c-9ac0eb2eb886-service-ca-bundle\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.247852 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.247711 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fzsqs\" (UniqueName: \"kubernetes.io/projected/1493c956-abd3-4deb-be9c-9ac0eb2eb886-kube-api-access-fzsqs\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.247852 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.247758 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1493c956-abd3-4deb-be9c-9ac0eb2eb886-serving-cert\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.247852 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.247813 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1493c956-abd3-4deb-be9c-9ac0eb2eb886-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.248246 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.248221 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1493c956-abd3-4deb-be9c-9ac0eb2eb886-tmp\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.248246 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.248238 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1493c956-abd3-4deb-be9c-9ac0eb2eb886-snapshots\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.248338 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.248317 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1493c956-abd3-4deb-be9c-9ac0eb2eb886-service-ca-bundle\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.248605 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.248588 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1493c956-abd3-4deb-be9c-9ac0eb2eb886-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.250104 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.250087 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1493c956-abd3-4deb-be9c-9ac0eb2eb886-serving-cert\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.257093 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.257066 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzsqs\" (UniqueName: \"kubernetes.io/projected/1493c956-abd3-4deb-be9c-9ac0eb2eb886-kube-api-access-fzsqs\") pod \"insights-operator-788bf58df5-kgq9t\" (UID: \"1493c956-abd3-4deb-be9c-9ac0eb2eb886\") " pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.366709 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.366669 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-788bf58df5-kgq9t" Jun 02 19:57:04.488919 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.488882 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-788bf58df5-kgq9t"] Jun 02 19:57:04.492150 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:04.492123 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1493c956_abd3_4deb_be9c_9ac0eb2eb886.slice/crio-25c95714769ae4601e826371601e29b066f756266a9d3470a212d272b0983fe5 WatchSource:0}: Error finding container 25c95714769ae4601e826371601e29b066f756266a9d3470a212d272b0983fe5: Status 404 returned error can't find the container with id 25c95714769ae4601e826371601e29b066f756266a9d3470a212d272b0983fe5 Jun 02 19:57:04.612389 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.612297 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-kgq9t" event={"ID":"1493c956-abd3-4deb-be9c-9ac0eb2eb886","Type":"ContainerStarted","Data":"25c95714769ae4601e826371601e29b066f756266a9d3470a212d272b0983fe5"} Jun 02 19:57:04.650734 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:04.650703 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:04.650883 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:04.650849 2548 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 19:57:04.650927 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:04.650917 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls podName:cc66e3d3-9282-4836-af26-6438bb0179dc nodeName:}" failed. No retries permitted until 2026-06-02 19:57:05.650899022 +0000 UTC m=+112.968112271 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-z57gd" (UID: "cc66e3d3-9282-4836-af26-6438bb0179dc") : secret "cluster-monitoring-operator-tls" not found Jun 02 19:57:05.658651 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:05.658600 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:05.659147 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:05.658795 2548 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 19:57:05.659147 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:05.658874 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls podName:cc66e3d3-9282-4836-af26-6438bb0179dc nodeName:}" failed. No retries permitted until 2026-06-02 19:57:07.658854352 +0000 UTC m=+114.976067603 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-z57gd" (UID: "cc66e3d3-9282-4836-af26-6438bb0179dc") : secret "cluster-monitoring-operator-tls" not found Jun 02 19:57:05.874318 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:05.874284 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7"] Jun 02 19:57:05.879101 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:05.879072 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:05.881681 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:05.881643 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 02 19:57:05.881835 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:05.881714 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:57:05.882618 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:05.882594 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-td4m7\"" Jun 02 19:57:05.882762 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:05.882594 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 02 19:57:05.887681 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:05.887660 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7"] Jun 02 19:57:05.961906 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:05.961733 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-cwmm7\" (UID: \"145475bc-a24b-4182-881c-5b7ba88a75c0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:05.961906 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:05.961805 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfhdc\" (UniqueName: \"kubernetes.io/projected/145475bc-a24b-4182-881c-5b7ba88a75c0-kube-api-access-pfhdc\") pod \"cluster-samples-operator-755bbf9bfb-cwmm7\" (UID: \"145475bc-a24b-4182-881c-5b7ba88a75c0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:06.062961 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:06.062920 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-cwmm7\" (UID: \"145475bc-a24b-4182-881c-5b7ba88a75c0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:06.062961 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:06.062963 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pfhdc\" (UniqueName: \"kubernetes.io/projected/145475bc-a24b-4182-881c-5b7ba88a75c0-kube-api-access-pfhdc\") pod \"cluster-samples-operator-755bbf9bfb-cwmm7\" (UID: \"145475bc-a24b-4182-881c-5b7ba88a75c0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:06.063274 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:06.063086 2548 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 19:57:06.063274 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:06.063175 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls podName:145475bc-a24b-4182-881c-5b7ba88a75c0 nodeName:}" failed. No retries permitted until 2026-06-02 19:57:06.563156967 +0000 UTC m=+113.880370199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-cwmm7" (UID: "145475bc-a24b-4182-881c-5b7ba88a75c0") : secret "samples-operator-tls" not found Jun 02 19:57:06.072512 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:06.072486 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfhdc\" (UniqueName: \"kubernetes.io/projected/145475bc-a24b-4182-881c-5b7ba88a75c0-kube-api-access-pfhdc\") pod \"cluster-samples-operator-755bbf9bfb-cwmm7\" (UID: \"145475bc-a24b-4182-881c-5b7ba88a75c0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:06.568468 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:06.568404 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-cwmm7\" (UID: \"145475bc-a24b-4182-881c-5b7ba88a75c0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:06.568679 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:06.568573 2548 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 19:57:06.568679 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:06.568654 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls podName:145475bc-a24b-4182-881c-5b7ba88a75c0 nodeName:}" failed. No retries permitted until 2026-06-02 19:57:07.568634019 +0000 UTC m=+114.885847266 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-cwmm7" (UID: "145475bc-a24b-4182-881c-5b7ba88a75c0") : secret "samples-operator-tls" not found Jun 02 19:57:06.617672 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:06.617577 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-kgq9t" event={"ID":"1493c956-abd3-4deb-be9c-9ac0eb2eb886","Type":"ContainerStarted","Data":"3072b4bcbd57f98d281380fbf4848efdfdc11fb08173b57127b1fe8bd719db7d"} Jun 02 19:57:06.635685 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:06.635635 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-788bf58df5-kgq9t" podStartSLOduration=0.765557234 podStartE2EDuration="2.635619519s" podCreationTimestamp="2026-06-02 19:57:04 +0000 UTC" firstStartedPulling="2026-06-02 19:57:04.494065009 +0000 UTC m=+111.811278245" lastFinishedPulling="2026-06-02 19:57:06.364127299 +0000 UTC m=+113.681340530" observedRunningTime="2026-06-02 19:57:06.634809503 +0000 UTC m=+113.952022758" watchObservedRunningTime="2026-06-02 19:57:06.635619519 +0000 UTC m=+113.952832772" Jun 02 19:57:07.577067 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:07.577024 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-cwmm7\" (UID: \"145475bc-a24b-4182-881c-5b7ba88a75c0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:07.577538 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:07.577192 2548 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 19:57:07.577538 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:07.577281 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls podName:145475bc-a24b-4182-881c-5b7ba88a75c0 nodeName:}" failed. No retries permitted until 2026-06-02 19:57:09.577259449 +0000 UTC m=+116.894472696 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-cwmm7" (UID: "145475bc-a24b-4182-881c-5b7ba88a75c0") : secret "samples-operator-tls" not found Jun 02 19:57:07.678022 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:07.677971 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:07.678201 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:07.678118 2548 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 19:57:07.678201 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:07.678186 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls podName:cc66e3d3-9282-4836-af26-6438bb0179dc nodeName:}" failed. No retries permitted until 2026-06-02 19:57:11.678169065 +0000 UTC m=+118.995382299 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-z57gd" (UID: "cc66e3d3-9282-4836-af26-6438bb0179dc") : secret "cluster-monitoring-operator-tls" not found Jun 02 19:57:09.593733 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:09.593678 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-cwmm7\" (UID: \"145475bc-a24b-4182-881c-5b7ba88a75c0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:09.594149 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:09.593853 2548 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 19:57:09.594149 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:09.593925 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls podName:145475bc-a24b-4182-881c-5b7ba88a75c0 nodeName:}" failed. No retries permitted until 2026-06-02 19:57:13.593908669 +0000 UTC m=+120.911121918 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-cwmm7" (UID: "145475bc-a24b-4182-881c-5b7ba88a75c0") : secret "samples-operator-tls" not found Jun 02 19:57:10.088694 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:10.088665 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lz2mg_c3674c37-2e21-4799-9545-d69c1985b721/dns-node-resolver/0.log" Jun 02 19:57:10.888940 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:10.888910 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kcv4k_861f1595-8ced-49bc-b4e3-2db727f3bb25/node-ca/0.log" Jun 02 19:57:10.954169 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:10.954136 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7"] Jun 02 19:57:10.957663 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:10.957647 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" Jun 02 19:57:10.962082 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:10.961901 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 02 19:57:10.962082 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:10.961926 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:57:10.962082 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:10.961940 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 02 19:57:10.962082 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:10.961947 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jddkc\"" Jun 02 19:57:10.962082 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:10.961971 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 02 19:57:10.966947 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:10.966928 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7"] Jun 02 19:57:11.106944 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.106899 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/76e3be1c-425f-407d-8805-bc27d505f922-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-khsh7\" (UID: \"76e3be1c-425f-407d-8805-bc27d505f922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" Jun 02 19:57:11.106944 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.106945 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h92t4\" (UniqueName: \"kubernetes.io/projected/76e3be1c-425f-407d-8805-bc27d505f922-kube-api-access-h92t4\") pod \"kube-storage-version-migrator-operator-96ddc787c-khsh7\" (UID: \"76e3be1c-425f-407d-8805-bc27d505f922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" Jun 02 19:57:11.107184 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.107068 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/76e3be1c-425f-407d-8805-bc27d505f922-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-khsh7\" (UID: \"76e3be1c-425f-407d-8805-bc27d505f922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" Jun 02 19:57:11.208433 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.208394 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/76e3be1c-425f-407d-8805-bc27d505f922-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-khsh7\" (UID: \"76e3be1c-425f-407d-8805-bc27d505f922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" Jun 02 19:57:11.208618 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.208440 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h92t4\" (UniqueName: \"kubernetes.io/projected/76e3be1c-425f-407d-8805-bc27d505f922-kube-api-access-h92t4\") pod \"kube-storage-version-migrator-operator-96ddc787c-khsh7\" (UID: \"76e3be1c-425f-407d-8805-bc27d505f922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" Jun 02 19:57:11.208618 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.208506 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/76e3be1c-425f-407d-8805-bc27d505f922-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-khsh7\" (UID: \"76e3be1c-425f-407d-8805-bc27d505f922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" Jun 02 19:57:11.209061 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.209039 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/76e3be1c-425f-407d-8805-bc27d505f922-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-khsh7\" (UID: \"76e3be1c-425f-407d-8805-bc27d505f922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" Jun 02 19:57:11.210642 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.210623 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/76e3be1c-425f-407d-8805-bc27d505f922-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-khsh7\" (UID: \"76e3be1c-425f-407d-8805-bc27d505f922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" Jun 02 19:57:11.220945 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.220920 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h92t4\" (UniqueName: \"kubernetes.io/projected/76e3be1c-425f-407d-8805-bc27d505f922-kube-api-access-h92t4\") pod \"kube-storage-version-migrator-operator-96ddc787c-khsh7\" (UID: \"76e3be1c-425f-407d-8805-bc27d505f922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" Jun 02 19:57:11.269013 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.268977 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" Jun 02 19:57:11.396774 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.396725 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7"] Jun 02 19:57:11.399839 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:11.399810 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76e3be1c_425f_407d_8805_bc27d505f922.slice/crio-c7c3af321bae190978d3ec58df36ef77f6837c3594910786ca4d2a88bd57ccea WatchSource:0}: Error finding container c7c3af321bae190978d3ec58df36ef77f6837c3594910786ca4d2a88bd57ccea: Status 404 returned error can't find the container with id c7c3af321bae190978d3ec58df36ef77f6837c3594910786ca4d2a88bd57ccea Jun 02 19:57:11.628586 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.628498 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" event={"ID":"76e3be1c-425f-407d-8805-bc27d505f922","Type":"ContainerStarted","Data":"c7c3af321bae190978d3ec58df36ef77f6837c3594910786ca4d2a88bd57ccea"} Jun 02 19:57:11.712665 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:11.712625 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:11.712860 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:11.712808 2548 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 19:57:11.712913 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:11.712892 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls podName:cc66e3d3-9282-4836-af26-6438bb0179dc nodeName:}" failed. No retries permitted until 2026-06-02 19:57:19.712872809 +0000 UTC m=+127.030086062 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-z57gd" (UID: "cc66e3d3-9282-4836-af26-6438bb0179dc") : secret "cluster-monitoring-operator-tls" not found Jun 02 19:57:13.627212 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:13.627166 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-cwmm7\" (UID: \"145475bc-a24b-4182-881c-5b7ba88a75c0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:13.627607 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:13.627332 2548 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 19:57:13.627607 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:13.627401 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls podName:145475bc-a24b-4182-881c-5b7ba88a75c0 nodeName:}" failed. No retries permitted until 2026-06-02 19:57:21.62738387 +0000 UTC m=+128.944597103 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-cwmm7" (UID: "145475bc-a24b-4182-881c-5b7ba88a75c0") : secret "samples-operator-tls" not found Jun 02 19:57:13.633852 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:13.633814 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" event={"ID":"76e3be1c-425f-407d-8805-bc27d505f922","Type":"ContainerStarted","Data":"179f38cde7bba363bbb124ad7906f2ab6faebae6f01e64d3ceec4f104d70ea2e"} Jun 02 19:57:13.665119 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:13.665057 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" podStartSLOduration=1.617353673 podStartE2EDuration="3.665039578s" podCreationTimestamp="2026-06-02 19:57:10 +0000 UTC" firstStartedPulling="2026-06-02 19:57:11.402149043 +0000 UTC m=+118.719362278" lastFinishedPulling="2026-06-02 19:57:13.449834948 +0000 UTC m=+120.767048183" observedRunningTime="2026-06-02 19:57:13.664915009 +0000 UTC m=+120.982128263" watchObservedRunningTime="2026-06-02 19:57:13.665039578 +0000 UTC m=+120.982252838" Jun 02 19:57:14.000553 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.000514 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9"] Jun 02 19:57:14.003145 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.003128 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" Jun 02 19:57:14.010783 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.010759 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 02 19:57:14.011545 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.011525 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:57:14.011655 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.011630 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 02 19:57:14.011799 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.011684 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-lzh7t\"" Jun 02 19:57:14.012396 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.012381 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 02 19:57:14.028523 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.028490 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9"] Jun 02 19:57:14.111646 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.111615 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-2lf95"] Jun 02 19:57:14.113508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.113491 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-2lf95" Jun 02 19:57:14.118607 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.118577 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2xsq7\"" Jun 02 19:57:14.129226 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.129199 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-2lf95"] Jun 02 19:57:14.132753 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.132716 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g46nx\" (UniqueName: \"kubernetes.io/projected/5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f-kube-api-access-g46nx\") pod \"service-ca-operator-69877444c-7hmx9\" (UID: \"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" Jun 02 19:57:14.132873 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.132804 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f-config\") pod \"service-ca-operator-69877444c-7hmx9\" (UID: \"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" Jun 02 19:57:14.132873 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.132833 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f-serving-cert\") pod \"service-ca-operator-69877444c-7hmx9\" (UID: \"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" Jun 02 19:57:14.233955 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.233911 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g46nx\" (UniqueName: \"kubernetes.io/projected/5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f-kube-api-access-g46nx\") pod \"service-ca-operator-69877444c-7hmx9\" (UID: \"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" Jun 02 19:57:14.234164 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.233964 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f-config\") pod \"service-ca-operator-69877444c-7hmx9\" (UID: \"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" Jun 02 19:57:14.234164 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.233994 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f-serving-cert\") pod \"service-ca-operator-69877444c-7hmx9\" (UID: \"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" Jun 02 19:57:14.234164 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.234039 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txcdn\" (UniqueName: \"kubernetes.io/projected/298466bf-240f-43b4-8d63-5dfb628db4ba-kube-api-access-txcdn\") pod \"network-check-source-56d84b9f44-2lf95\" (UID: \"298466bf-240f-43b4-8d63-5dfb628db4ba\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-2lf95" Jun 02 19:57:14.234524 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.234504 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f-config\") pod \"service-ca-operator-69877444c-7hmx9\" (UID: \"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" Jun 02 19:57:14.236243 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.236223 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f-serving-cert\") pod \"service-ca-operator-69877444c-7hmx9\" (UID: \"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" Jun 02 19:57:14.248194 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.248162 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g46nx\" (UniqueName: \"kubernetes.io/projected/5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f-kube-api-access-g46nx\") pod \"service-ca-operator-69877444c-7hmx9\" (UID: \"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" Jun 02 19:57:14.312057 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.311953 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" Jun 02 19:57:14.335051 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.335019 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-txcdn\" (UniqueName: \"kubernetes.io/projected/298466bf-240f-43b4-8d63-5dfb628db4ba-kube-api-access-txcdn\") pod \"network-check-source-56d84b9f44-2lf95\" (UID: \"298466bf-240f-43b4-8d63-5dfb628db4ba\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-2lf95" Jun 02 19:57:14.346361 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.346328 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txcdn\" (UniqueName: \"kubernetes.io/projected/298466bf-240f-43b4-8d63-5dfb628db4ba-kube-api-access-txcdn\") pod \"network-check-source-56d84b9f44-2lf95\" (UID: \"298466bf-240f-43b4-8d63-5dfb628db4ba\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-2lf95" Jun 02 19:57:14.422066 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.422028 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-2lf95" Jun 02 19:57:14.459492 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.459458 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9"] Jun 02 19:57:14.463350 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:14.463318 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ab1f9fc_c1b0_419a_a8c9_ebc565a5e35f.slice/crio-bfa97559defe3e9611981be62d5a08a2e23280abbdceebb07745b928b0706ddf WatchSource:0}: Error finding container bfa97559defe3e9611981be62d5a08a2e23280abbdceebb07745b928b0706ddf: Status 404 returned error can't find the container with id bfa97559defe3e9611981be62d5a08a2e23280abbdceebb07745b928b0706ddf Jun 02 19:57:14.595832 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.595734 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-2lf95"] Jun 02 19:57:14.599193 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:14.599166 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod298466bf_240f_43b4_8d63_5dfb628db4ba.slice/crio-d9cb1a32d31751027c89455534efe3d8e671d653afc62ab3691d9f0c29c54ff4 WatchSource:0}: Error finding container d9cb1a32d31751027c89455534efe3d8e671d653afc62ab3691d9f0c29c54ff4: Status 404 returned error can't find the container with id d9cb1a32d31751027c89455534efe3d8e671d653afc62ab3691d9f0c29c54ff4 Jun 02 19:57:14.637296 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.637257 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-2lf95" event={"ID":"298466bf-240f-43b4-8d63-5dfb628db4ba","Type":"ContainerStarted","Data":"d9cb1a32d31751027c89455534efe3d8e671d653afc62ab3691d9f0c29c54ff4"} Jun 02 19:57:14.638397 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:14.638371 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" event={"ID":"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f","Type":"ContainerStarted","Data":"bfa97559defe3e9611981be62d5a08a2e23280abbdceebb07745b928b0706ddf"} Jun 02 19:57:15.643500 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:15.643467 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-2lf95" event={"ID":"298466bf-240f-43b4-8d63-5dfb628db4ba","Type":"ContainerStarted","Data":"f7500c45688fa2f2fe2a52ce8980f0021ece0cc95c4812e43343fe25038eafda"} Jun 02 19:57:15.669418 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:15.669365 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-2lf95" podStartSLOduration=1.669350493 podStartE2EDuration="1.669350493s" podCreationTimestamp="2026-06-02 19:57:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:57:15.668204651 +0000 UTC m=+122.985417907" watchObservedRunningTime="2026-06-02 19:57:15.669350493 +0000 UTC m=+122.986563747" Jun 02 19:57:17.650827 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:17.650786 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" event={"ID":"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f","Type":"ContainerStarted","Data":"742f5d0a2dac9584701d83515e20366f83f74dce7b12ea3e93ab1d6724f3855e"} Jun 02 19:57:17.675493 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:17.675436 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" podStartSLOduration=2.454431747 podStartE2EDuration="4.6754225s" podCreationTimestamp="2026-06-02 19:57:13 +0000 UTC" firstStartedPulling="2026-06-02 19:57:14.465707033 +0000 UTC m=+121.782920269" lastFinishedPulling="2026-06-02 19:57:16.686697771 +0000 UTC m=+124.003911022" observedRunningTime="2026-06-02 19:57:17.674680836 +0000 UTC m=+124.991894087" watchObservedRunningTime="2026-06-02 19:57:17.6754225 +0000 UTC m=+124.992635754" Jun 02 19:57:19.779821 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:19.779761 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:19.780326 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:19.779938 2548 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 19:57:19.780326 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:19.780044 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls podName:cc66e3d3-9282-4836-af26-6438bb0179dc nodeName:}" failed. No retries permitted until 2026-06-02 19:57:35.780022485 +0000 UTC m=+143.097235727 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-z57gd" (UID: "cc66e3d3-9282-4836-af26-6438bb0179dc") : secret "cluster-monitoring-operator-tls" not found Jun 02 19:57:20.275044 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.275006 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-685df94d57-z8qhh"] Jun 02 19:57:20.276982 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.276967 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-z8qhh" Jun 02 19:57:20.280641 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.280611 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 02 19:57:20.280641 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.280616 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-pd7vw\"" Jun 02 19:57:20.281573 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.281556 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 02 19:57:20.281660 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.281643 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 02 19:57:20.281724 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.281687 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 02 19:57:20.287413 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.287389 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-z8qhh"] Jun 02 19:57:20.384327 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.384291 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/084a91a6-eb74-4553-ba4d-e158ab55a909-signing-cabundle\") pod \"service-ca-685df94d57-z8qhh\" (UID: \"084a91a6-eb74-4553-ba4d-e158ab55a909\") " pod="openshift-service-ca/service-ca-685df94d57-z8qhh" Jun 02 19:57:20.384516 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.384335 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/084a91a6-eb74-4553-ba4d-e158ab55a909-signing-key\") pod \"service-ca-685df94d57-z8qhh\" (UID: \"084a91a6-eb74-4553-ba4d-e158ab55a909\") " pod="openshift-service-ca/service-ca-685df94d57-z8qhh" Jun 02 19:57:20.384516 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.384450 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cphwc\" (UniqueName: \"kubernetes.io/projected/084a91a6-eb74-4553-ba4d-e158ab55a909-kube-api-access-cphwc\") pod \"service-ca-685df94d57-z8qhh\" (UID: \"084a91a6-eb74-4553-ba4d-e158ab55a909\") " pod="openshift-service-ca/service-ca-685df94d57-z8qhh" Jun 02 19:57:20.485958 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.485906 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/084a91a6-eb74-4553-ba4d-e158ab55a909-signing-cabundle\") pod \"service-ca-685df94d57-z8qhh\" (UID: \"084a91a6-eb74-4553-ba4d-e158ab55a909\") " pod="openshift-service-ca/service-ca-685df94d57-z8qhh" Jun 02 19:57:20.486156 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.485965 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/084a91a6-eb74-4553-ba4d-e158ab55a909-signing-key\") pod \"service-ca-685df94d57-z8qhh\" (UID: \"084a91a6-eb74-4553-ba4d-e158ab55a909\") " pod="openshift-service-ca/service-ca-685df94d57-z8qhh" Jun 02 19:57:20.486156 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.486036 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cphwc\" (UniqueName: \"kubernetes.io/projected/084a91a6-eb74-4553-ba4d-e158ab55a909-kube-api-access-cphwc\") pod \"service-ca-685df94d57-z8qhh\" (UID: \"084a91a6-eb74-4553-ba4d-e158ab55a909\") " pod="openshift-service-ca/service-ca-685df94d57-z8qhh" Jun 02 19:57:20.486570 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.486548 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/084a91a6-eb74-4553-ba4d-e158ab55a909-signing-cabundle\") pod \"service-ca-685df94d57-z8qhh\" (UID: \"084a91a6-eb74-4553-ba4d-e158ab55a909\") " pod="openshift-service-ca/service-ca-685df94d57-z8qhh" Jun 02 19:57:20.488460 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.488437 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/084a91a6-eb74-4553-ba4d-e158ab55a909-signing-key\") pod \"service-ca-685df94d57-z8qhh\" (UID: \"084a91a6-eb74-4553-ba4d-e158ab55a909\") " pod="openshift-service-ca/service-ca-685df94d57-z8qhh" Jun 02 19:57:20.494500 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.494474 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cphwc\" (UniqueName: \"kubernetes.io/projected/084a91a6-eb74-4553-ba4d-e158ab55a909-kube-api-access-cphwc\") pod \"service-ca-685df94d57-z8qhh\" (UID: \"084a91a6-eb74-4553-ba4d-e158ab55a909\") " pod="openshift-service-ca/service-ca-685df94d57-z8qhh" Jun 02 19:57:20.585643 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.585548 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-z8qhh" Jun 02 19:57:20.715395 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:20.715350 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-z8qhh"] Jun 02 19:57:20.721306 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:20.721267 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod084a91a6_eb74_4553_ba4d_e158ab55a909.slice/crio-830da29fa5ae5015b7cb090b3f79b68dc8be5182c0492ee680ce4bb434c1de03 WatchSource:0}: Error finding container 830da29fa5ae5015b7cb090b3f79b68dc8be5182c0492ee680ce4bb434c1de03: Status 404 returned error can't find the container with id 830da29fa5ae5015b7cb090b3f79b68dc8be5182c0492ee680ce4bb434c1de03 Jun 02 19:57:21.661406 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:21.661370 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-z8qhh" event={"ID":"084a91a6-eb74-4553-ba4d-e158ab55a909","Type":"ContainerStarted","Data":"2bd5a38123d8bdff35a8204a7cacc4d6a95eb6545e9e1cf328f2957e42ee7876"} Jun 02 19:57:21.661406 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:21.661407 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-z8qhh" event={"ID":"084a91a6-eb74-4553-ba4d-e158ab55a909","Type":"ContainerStarted","Data":"830da29fa5ae5015b7cb090b3f79b68dc8be5182c0492ee680ce4bb434c1de03"} Jun 02 19:57:21.695508 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:21.695475 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-cwmm7\" (UID: \"145475bc-a24b-4182-881c-5b7ba88a75c0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:21.695658 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:21.695625 2548 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 19:57:21.695698 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:21.695686 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls podName:145475bc-a24b-4182-881c-5b7ba88a75c0 nodeName:}" failed. No retries permitted until 2026-06-02 19:57:37.695669571 +0000 UTC m=+145.012882809 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-cwmm7" (UID: "145475bc-a24b-4182-881c-5b7ba88a75c0") : secret "samples-operator-tls" not found Jun 02 19:57:23.109657 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:23.109614 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:57:23.110108 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:23.109735 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 19:57:23.110108 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:23.109814 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs podName:c848a2cb-1edf-4aa2-b593-c0f19d847476 nodeName:}" failed. No retries permitted until 2026-06-02 19:59:25.109799694 +0000 UTC m=+252.427012926 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs") pod "network-metrics-daemon-wwpfn" (UID: "c848a2cb-1edf-4aa2-b593-c0f19d847476") : secret "metrics-daemon-secret" not found Jun 02 19:57:35.812586 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:35.812542 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:35.814983 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:35.814961 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc66e3d3-9282-4836-af26-6438bb0179dc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-z57gd\" (UID: \"cc66e3d3-9282-4836-af26-6438bb0179dc\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:36.069193 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:36.069109 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-fbhzp\"" Jun 02 19:57:36.077289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:36.077259 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" Jun 02 19:57:36.206730 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:36.206661 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-685df94d57-z8qhh" podStartSLOduration=16.206640452 podStartE2EDuration="16.206640452s" podCreationTimestamp="2026-06-02 19:57:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:57:21.692191649 +0000 UTC m=+129.009404905" watchObservedRunningTime="2026-06-02 19:57:36.206640452 +0000 UTC m=+143.523853706" Jun 02 19:57:36.207639 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:36.207617 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd"] Jun 02 19:57:36.210353 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:36.210317 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc66e3d3_9282_4836_af26_6438bb0179dc.slice/crio-a55f855aa184c52b9f8c0d96b07dffbdb7785eae771336c87d49ef63153204fb WatchSource:0}: Error finding container a55f855aa184c52b9f8c0d96b07dffbdb7785eae771336c87d49ef63153204fb: Status 404 returned error can't find the container with id a55f855aa184c52b9f8c0d96b07dffbdb7785eae771336c87d49ef63153204fb Jun 02 19:57:36.695713 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:36.695675 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" event={"ID":"cc66e3d3-9282-4836-af26-6438bb0179dc","Type":"ContainerStarted","Data":"a55f855aa184c52b9f8c0d96b07dffbdb7785eae771336c87d49ef63153204fb"} Jun 02 19:57:37.727682 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:37.727632 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-cwmm7\" (UID: \"145475bc-a24b-4182-881c-5b7ba88a75c0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:37.730497 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:37.730471 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/145475bc-a24b-4182-881c-5b7ba88a75c0-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-cwmm7\" (UID: \"145475bc-a24b-4182-881c-5b7ba88a75c0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:37.992421 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:37.992356 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-td4m7\"" Jun 02 19:57:37.999826 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:37.999808 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" Jun 02 19:57:38.136805 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:38.136767 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7"] Jun 02 19:57:38.703230 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:38.703195 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" event={"ID":"cc66e3d3-9282-4836-af26-6438bb0179dc","Type":"ContainerStarted","Data":"488f9332aa9141a383d3aa520ae07b0978c4dcee45462c5567b6784f52852ba5"} Jun 02 19:57:38.704208 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:38.704181 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" event={"ID":"145475bc-a24b-4182-881c-5b7ba88a75c0","Type":"ContainerStarted","Data":"61c9c3b7882c1a2dd541c0dd4880af7e23f0aa8f1703e4fdfca97b7bfcce5467"} Jun 02 19:57:40.711133 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:40.711095 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" event={"ID":"145475bc-a24b-4182-881c-5b7ba88a75c0","Type":"ContainerStarted","Data":"93fbc39cf40af394d0ab218ed02ed88aced596eb67a6ecd49fb64e8ea0d023cb"} Jun 02 19:57:40.711512 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:40.711140 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" event={"ID":"145475bc-a24b-4182-881c-5b7ba88a75c0","Type":"ContainerStarted","Data":"9969703670e0306a6965831cbf1c74ea724ceacaeab96d087e53aa2fc5b2aa46"} Jun 02 19:57:40.730234 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:40.730186 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-z57gd" podStartSLOduration=35.944147303 podStartE2EDuration="37.730173167s" podCreationTimestamp="2026-06-02 19:57:03 +0000 UTC" firstStartedPulling="2026-06-02 19:57:36.212160589 +0000 UTC m=+143.529373821" lastFinishedPulling="2026-06-02 19:57:37.998186447 +0000 UTC m=+145.315399685" observedRunningTime="2026-06-02 19:57:38.724691977 +0000 UTC m=+146.041905232" watchObservedRunningTime="2026-06-02 19:57:40.730173167 +0000 UTC m=+148.047386421" Jun 02 19:57:43.416640 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.416571 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-cwmm7" podStartSLOduration=36.790722704 podStartE2EDuration="38.416548612s" podCreationTimestamp="2026-06-02 19:57:05 +0000 UTC" firstStartedPulling="2026-06-02 19:57:38.183290813 +0000 UTC m=+145.500504051" lastFinishedPulling="2026-06-02 19:57:39.809116719 +0000 UTC m=+147.126329959" observedRunningTime="2026-06-02 19:57:40.729727333 +0000 UTC m=+148.046940586" watchObservedRunningTime="2026-06-02 19:57:43.416548612 +0000 UTC m=+150.733761867" Jun 02 19:57:43.417590 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.417570 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tmzjz"] Jun 02 19:57:43.419751 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.419723 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.423000 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.422969 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jnsrc\"" Jun 02 19:57:43.423975 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.423953 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 19:57:43.424874 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.424854 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 19:57:43.444079 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.444047 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tmzjz"] Jun 02 19:57:43.474015 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.473979 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c332bbad-564f-4d40-84c6-4fff506f49eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.474015 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.474019 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c332bbad-564f-4d40-84c6-4fff506f49eb-crio-socket\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.474272 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.474048 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hx6h\" (UniqueName: \"kubernetes.io/projected/c332bbad-564f-4d40-84c6-4fff506f49eb-kube-api-access-5hx6h\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.474272 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.474109 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c332bbad-564f-4d40-84c6-4fff506f49eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.474272 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.474131 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c332bbad-564f-4d40-84c6-4fff506f49eb-data-volume\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.574721 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.574679 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c332bbad-564f-4d40-84c6-4fff506f49eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.574721 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.574724 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c332bbad-564f-4d40-84c6-4fff506f49eb-data-volume\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.574961 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.574828 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c332bbad-564f-4d40-84c6-4fff506f49eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.574961 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.574851 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c332bbad-564f-4d40-84c6-4fff506f49eb-crio-socket\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.574961 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.574877 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5hx6h\" (UniqueName: \"kubernetes.io/projected/c332bbad-564f-4d40-84c6-4fff506f49eb-kube-api-access-5hx6h\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.575137 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.574987 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c332bbad-564f-4d40-84c6-4fff506f49eb-crio-socket\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.575179 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.575165 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c332bbad-564f-4d40-84c6-4fff506f49eb-data-volume\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.575364 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.575346 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c332bbad-564f-4d40-84c6-4fff506f49eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.577302 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.577278 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c332bbad-564f-4d40-84c6-4fff506f49eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.584125 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.584102 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hx6h\" (UniqueName: \"kubernetes.io/projected/c332bbad-564f-4d40-84c6-4fff506f49eb-kube-api-access-5hx6h\") pod \"insights-runtime-extractor-tmzjz\" (UID: \"c332bbad-564f-4d40-84c6-4fff506f49eb\") " pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.729461 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.729375 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tmzjz" Jun 02 19:57:43.877129 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:43.877087 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tmzjz"] Jun 02 19:57:43.881733 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:43.881698 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc332bbad_564f_4d40_84c6_4fff506f49eb.slice/crio-dc7ca6df14483bf79f699e90f4a7411bc42d1e4f17e85a9a959189b60eb2f5b4 WatchSource:0}: Error finding container dc7ca6df14483bf79f699e90f4a7411bc42d1e4f17e85a9a959189b60eb2f5b4: Status 404 returned error can't find the container with id dc7ca6df14483bf79f699e90f4a7411bc42d1e4f17e85a9a959189b60eb2f5b4 Jun 02 19:57:44.721454 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:44.721413 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmzjz" event={"ID":"c332bbad-564f-4d40-84c6-4fff506f49eb","Type":"ContainerStarted","Data":"918699d8a1f2e6514276d74a02190f2b2acbd738de7a6443bda4d26de3038e05"} Jun 02 19:57:44.721454 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:44.721449 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmzjz" event={"ID":"c332bbad-564f-4d40-84c6-4fff506f49eb","Type":"ContainerStarted","Data":"20e2338f613b55fd812a02d113ed77ca47a11ef1c45e19508ed26d0ba2ec4bd6"} Jun 02 19:57:44.721454 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:44.721458 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmzjz" event={"ID":"c332bbad-564f-4d40-84c6-4fff506f49eb","Type":"ContainerStarted","Data":"dc7ca6df14483bf79f699e90f4a7411bc42d1e4f17e85a9a959189b60eb2f5b4"} Jun 02 19:57:46.584630 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.584544 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l"] Jun 02 19:57:46.587110 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.587091 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.590540 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.590505 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-r67tb\"" Jun 02 19:57:46.590678 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.590549 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 02 19:57:46.591657 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.591638 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 02 19:57:46.591657 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.591648 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 19:57:46.596332 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.596308 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l"] Jun 02 19:57:46.700466 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.700413 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db78ab6d-7ab0-483d-a46f-98d4f03a07a3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-54g6l\" (UID: \"db78ab6d-7ab0-483d-a46f-98d4f03a07a3\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.700466 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.700472 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/db78ab6d-7ab0-483d-a46f-98d4f03a07a3-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-54g6l\" (UID: \"db78ab6d-7ab0-483d-a46f-98d4f03a07a3\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.700679 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.700501 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw8x8\" (UniqueName: \"kubernetes.io/projected/db78ab6d-7ab0-483d-a46f-98d4f03a07a3-kube-api-access-cw8x8\") pod \"prometheus-operator-658bcd8bdf-54g6l\" (UID: \"db78ab6d-7ab0-483d-a46f-98d4f03a07a3\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.700716 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.700677 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db78ab6d-7ab0-483d-a46f-98d4f03a07a3-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-54g6l\" (UID: \"db78ab6d-7ab0-483d-a46f-98d4f03a07a3\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.729477 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.729446 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmzjz" event={"ID":"c332bbad-564f-4d40-84c6-4fff506f49eb","Type":"ContainerStarted","Data":"211fc80558e5b6376ca8cae8012c02076d337c18dc64a6be34117a9c026d0d6d"} Jun 02 19:57:46.749126 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.749076 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tmzjz" podStartSLOduration=1.540346613 podStartE2EDuration="3.749061314s" podCreationTimestamp="2026-06-02 19:57:43 +0000 UTC" firstStartedPulling="2026-06-02 19:57:43.939602848 +0000 UTC m=+151.256816080" lastFinishedPulling="2026-06-02 19:57:46.148317545 +0000 UTC m=+153.465530781" observedRunningTime="2026-06-02 19:57:46.748034704 +0000 UTC m=+154.065247958" watchObservedRunningTime="2026-06-02 19:57:46.749061314 +0000 UTC m=+154.066274586" Jun 02 19:57:46.802031 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.801996 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db78ab6d-7ab0-483d-a46f-98d4f03a07a3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-54g6l\" (UID: \"db78ab6d-7ab0-483d-a46f-98d4f03a07a3\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.802188 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.802038 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/db78ab6d-7ab0-483d-a46f-98d4f03a07a3-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-54g6l\" (UID: \"db78ab6d-7ab0-483d-a46f-98d4f03a07a3\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.807474 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.802150 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cw8x8\" (UniqueName: \"kubernetes.io/projected/db78ab6d-7ab0-483d-a46f-98d4f03a07a3-kube-api-access-cw8x8\") pod \"prometheus-operator-658bcd8bdf-54g6l\" (UID: \"db78ab6d-7ab0-483d-a46f-98d4f03a07a3\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.807474 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.802699 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db78ab6d-7ab0-483d-a46f-98d4f03a07a3-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-54g6l\" (UID: \"db78ab6d-7ab0-483d-a46f-98d4f03a07a3\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.807474 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.804007 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db78ab6d-7ab0-483d-a46f-98d4f03a07a3-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-54g6l\" (UID: \"db78ab6d-7ab0-483d-a46f-98d4f03a07a3\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.807474 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.805369 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db78ab6d-7ab0-483d-a46f-98d4f03a07a3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-54g6l\" (UID: \"db78ab6d-7ab0-483d-a46f-98d4f03a07a3\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.807808 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.807522 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/db78ab6d-7ab0-483d-a46f-98d4f03a07a3-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-54g6l\" (UID: \"db78ab6d-7ab0-483d-a46f-98d4f03a07a3\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.811994 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.811967 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw8x8\" (UniqueName: \"kubernetes.io/projected/db78ab6d-7ab0-483d-a46f-98d4f03a07a3-kube-api-access-cw8x8\") pod \"prometheus-operator-658bcd8bdf-54g6l\" (UID: \"db78ab6d-7ab0-483d-a46f-98d4f03a07a3\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:46.897586 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:46.896760 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" Jun 02 19:57:47.030869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:47.030717 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l"] Jun 02 19:57:47.033450 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:47.033420 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb78ab6d_7ab0_483d_a46f_98d4f03a07a3.slice/crio-fb737c9976d7f15c732052af665e717deb3b6556fddd64c83e8ce048f971f166 WatchSource:0}: Error finding container fb737c9976d7f15c732052af665e717deb3b6556fddd64c83e8ce048f971f166: Status 404 returned error can't find the container with id fb737c9976d7f15c732052af665e717deb3b6556fddd64c83e8ce048f971f166 Jun 02 19:57:47.733160 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:47.733122 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" event={"ID":"db78ab6d-7ab0-483d-a46f-98d4f03a07a3","Type":"ContainerStarted","Data":"fb737c9976d7f15c732052af665e717deb3b6556fddd64c83e8ce048f971f166"} Jun 02 19:57:48.737227 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:48.737180 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" event={"ID":"db78ab6d-7ab0-483d-a46f-98d4f03a07a3","Type":"ContainerStarted","Data":"5e76f089f3a360ef3bb327de578d01bf855324838361b4beb2bd297ad92d8363"} Jun 02 19:57:48.737621 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:48.737237 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" event={"ID":"db78ab6d-7ab0-483d-a46f-98d4f03a07a3","Type":"ContainerStarted","Data":"c4bfc27d626e5536c2416073c8221480cdad8fdf39230891cae0cfe8a81af2bb"} Jun 02 19:57:48.755598 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:48.755544 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-54g6l" podStartSLOduration=1.635165695 podStartE2EDuration="2.755532379s" podCreationTimestamp="2026-06-02 19:57:46 +0000 UTC" firstStartedPulling="2026-06-02 19:57:47.035260954 +0000 UTC m=+154.352474186" lastFinishedPulling="2026-06-02 19:57:48.155627635 +0000 UTC m=+155.472840870" observedRunningTime="2026-06-02 19:57:48.753825517 +0000 UTC m=+156.071038781" watchObservedRunningTime="2026-06-02 19:57:48.755532379 +0000 UTC m=+156.072745632" Jun 02 19:57:50.555114 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:50.555066 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" podUID="d58c985a-62d6-4fa6-85f2-a79d3cebac80" Jun 02 19:57:50.596007 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:50.595965 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-jjfnh" podUID="3bade01b-4276-40ba-b81a-d607f88064e2" Jun 02 19:57:50.601126 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:50.601097 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rmbfk" podUID="7952dddc-f51b-4a0c-8a23-feecfec0457a" Jun 02 19:57:50.742331 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:50.742304 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rmbfk" Jun 02 19:57:50.742331 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:50.742317 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:57:51.141431 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.141386 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-r2brz"] Jun 02 19:57:51.144135 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.144112 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.146913 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.146889 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zt44n\"" Jun 02 19:57:51.147644 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.147627 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 02 19:57:51.147782 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.147763 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 02 19:57:51.158097 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.158067 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lqhbt"] Jun 02 19:57:51.161132 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.161108 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-r2brz"] Jun 02 19:57:51.161394 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.161367 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.164499 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.164295 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 19:57:51.165258 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.164781 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-978m5\"" Jun 02 19:57:51.165258 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.164975 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 19:57:51.165385 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.165295 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 19:57:51.181087 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.181055 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m"] Jun 02 19:57:51.183686 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.183666 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.188208 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.188023 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 02 19:57:51.188208 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.188023 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 02 19:57:51.188415 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.188297 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-h4t6s\"" Jun 02 19:57:51.188660 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.188640 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 02 19:57:51.208488 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.208453 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m"] Jun 02 19:57:51.239636 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.239601 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlblv\" (UniqueName: \"kubernetes.io/projected/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-kube-api-access-zlblv\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.239636 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.239639 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-textfile\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.239922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.239665 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.239922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.239693 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2115b62d-ce15-4d1b-8fce-7dbee21ace94-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-r2brz\" (UID: \"2115b62d-ce15-4d1b-8fce-7dbee21ace94\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.239922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.239784 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2115b62d-ce15-4d1b-8fce-7dbee21ace94-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-r2brz\" (UID: \"2115b62d-ce15-4d1b-8fce-7dbee21ace94\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.239922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.239804 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.239922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.239861 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5800e54-11e6-4369-ab41-1d8454b4caee-sys\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.239922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.239879 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.239922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.239905 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-wtmp\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.240259 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.239929 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-accelerators-collector-config\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.240259 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.240003 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-tls\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.240259 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.240086 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.240259 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.240113 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5800e54-11e6-4369-ab41-1d8454b4caee-metrics-client-ca\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.240259 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.240136 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vr7d\" (UniqueName: \"kubernetes.io/projected/f5800e54-11e6-4369-ab41-1d8454b4caee-kube-api-access-8vr7d\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.240259 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.240165 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2115b62d-ce15-4d1b-8fce-7dbee21ace94-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-r2brz\" (UID: \"2115b62d-ce15-4d1b-8fce-7dbee21ace94\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.240259 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.240192 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6sm2\" (UniqueName: \"kubernetes.io/projected/2115b62d-ce15-4d1b-8fce-7dbee21ace94-kube-api-access-g6sm2\") pod \"openshift-state-metrics-85658796db-r2brz\" (UID: \"2115b62d-ce15-4d1b-8fce-7dbee21ace94\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.240259 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.240228 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f5800e54-11e6-4369-ab41-1d8454b4caee-root\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.240664 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.240280 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.240664 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.240310 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.335874 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:51.335830 2548 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-wwpfn" podUID="c848a2cb-1edf-4aa2-b593-c0f19d847476" Jun 02 19:57:51.341163 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341123 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5800e54-11e6-4369-ab41-1d8454b4caee-sys\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.341322 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341176 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.341322 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341210 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-wtmp\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.341322 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341236 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5800e54-11e6-4369-ab41-1d8454b4caee-sys\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.341322 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341240 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-accelerators-collector-config\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.341524 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341342 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-tls\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.341524 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341400 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.341524 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341425 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5800e54-11e6-4369-ab41-1d8454b4caee-metrics-client-ca\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.341524 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341449 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8vr7d\" (UniqueName: \"kubernetes.io/projected/f5800e54-11e6-4369-ab41-1d8454b4caee-kube-api-access-8vr7d\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.341524 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341477 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2115b62d-ce15-4d1b-8fce-7dbee21ace94-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-r2brz\" (UID: \"2115b62d-ce15-4d1b-8fce-7dbee21ace94\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.341524 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341505 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g6sm2\" (UniqueName: \"kubernetes.io/projected/2115b62d-ce15-4d1b-8fce-7dbee21ace94-kube-api-access-g6sm2\") pod \"openshift-state-metrics-85658796db-r2brz\" (UID: \"2115b62d-ce15-4d1b-8fce-7dbee21ace94\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.341862 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341542 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f5800e54-11e6-4369-ab41-1d8454b4caee-root\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.341862 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:51.341579 2548 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 02 19:57:51.341862 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341583 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.341862 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341630 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.341862 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:57:51.341643 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-tls podName:f5800e54-11e6-4369-ab41-1d8454b4caee nodeName:}" failed. No retries permitted until 2026-06-02 19:57:51.841624228 +0000 UTC m=+159.158837463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-tls") pod "node-exporter-lqhbt" (UID: "f5800e54-11e6-4369-ab41-1d8454b4caee") : secret "node-exporter-tls" not found Jun 02 19:57:51.341862 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341693 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.341862 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341477 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-wtmp\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.342226 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341902 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-accelerators-collector-config\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.342226 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.341977 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f5800e54-11e6-4369-ab41-1d8454b4caee-root\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.342330 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.342240 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5800e54-11e6-4369-ab41-1d8454b4caee-metrics-client-ca\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.342330 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.342315 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zlblv\" (UniqueName: \"kubernetes.io/projected/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-kube-api-access-zlblv\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.342423 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.342351 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-textfile\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.342423 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.342379 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.342423 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.342410 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2115b62d-ce15-4d1b-8fce-7dbee21ace94-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-r2brz\" (UID: \"2115b62d-ce15-4d1b-8fce-7dbee21ace94\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.342565 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.342469 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2115b62d-ce15-4d1b-8fce-7dbee21ace94-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-r2brz\" (UID: \"2115b62d-ce15-4d1b-8fce-7dbee21ace94\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.342565 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.342494 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.343114 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.342704 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.343114 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.342980 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-textfile\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.343275 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.343154 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.343958 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.343930 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2115b62d-ce15-4d1b-8fce-7dbee21ace94-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-r2brz\" (UID: \"2115b62d-ce15-4d1b-8fce-7dbee21ace94\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.345083 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.345025 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.345288 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.345264 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.345769 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.345729 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.346191 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.346174 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2115b62d-ce15-4d1b-8fce-7dbee21ace94-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-r2brz\" (UID: \"2115b62d-ce15-4d1b-8fce-7dbee21ace94\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.346273 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.346207 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2115b62d-ce15-4d1b-8fce-7dbee21ace94-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-r2brz\" (UID: \"2115b62d-ce15-4d1b-8fce-7dbee21ace94\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.353242 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.353217 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vr7d\" (UniqueName: \"kubernetes.io/projected/f5800e54-11e6-4369-ab41-1d8454b4caee-kube-api-access-8vr7d\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.353955 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.353884 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlblv\" (UniqueName: \"kubernetes.io/projected/62c1ac6c-eef9-4d69-8835-b25a23d64e0c-kube-api-access-zlblv\") pod \"kube-state-metrics-8469c486bb-g8c8m\" (UID: \"62c1ac6c-eef9-4d69-8835-b25a23d64e0c\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.353955 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.353895 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6sm2\" (UniqueName: \"kubernetes.io/projected/2115b62d-ce15-4d1b-8fce-7dbee21ace94-kube-api-access-g6sm2\") pod \"openshift-state-metrics-85658796db-r2brz\" (UID: \"2115b62d-ce15-4d1b-8fce-7dbee21ace94\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.457525 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.457473 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" Jun 02 19:57:51.495654 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.495617 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" Jun 02 19:57:51.605667 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.605631 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-r2brz"] Jun 02 19:57:51.608542 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:51.608495 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2115b62d_ce15_4d1b_8fce_7dbee21ace94.slice/crio-51af0713127573dbecb78bf01311a70f5a0bf70ecde82d90a4c973e4387462c2 WatchSource:0}: Error finding container 51af0713127573dbecb78bf01311a70f5a0bf70ecde82d90a4c973e4387462c2: Status 404 returned error can't find the container with id 51af0713127573dbecb78bf01311a70f5a0bf70ecde82d90a4c973e4387462c2 Jun 02 19:57:51.657423 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.657397 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m"] Jun 02 19:57:51.660230 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:51.660190 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c1ac6c_eef9_4d69_8835_b25a23d64e0c.slice/crio-096486cab878aafc9303a45d996a9baffead66767ca2423843572527af328b00 WatchSource:0}: Error finding container 096486cab878aafc9303a45d996a9baffead66767ca2423843572527af328b00: Status 404 returned error can't find the container with id 096486cab878aafc9303a45d996a9baffead66767ca2423843572527af328b00 Jun 02 19:57:51.746984 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.746943 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" event={"ID":"2115b62d-ce15-4d1b-8fce-7dbee21ace94","Type":"ContainerStarted","Data":"2bfc2a18e8dcc1bc2a0ebb0a154ea66718b0c5dbbf50a23bb3418e76c552fd13"} Jun 02 19:57:51.746984 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.746994 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" event={"ID":"2115b62d-ce15-4d1b-8fce-7dbee21ace94","Type":"ContainerStarted","Data":"cdf94e0a4d6c615b827c0eacd56223c8ded02a003d09f38829d7f778028aa77f"} Jun 02 19:57:51.746984 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.747010 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" event={"ID":"2115b62d-ce15-4d1b-8fce-7dbee21ace94","Type":"ContainerStarted","Data":"51af0713127573dbecb78bf01311a70f5a0bf70ecde82d90a4c973e4387462c2"} Jun 02 19:57:51.748318 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.748291 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" event={"ID":"62c1ac6c-eef9-4d69-8835-b25a23d64e0c","Type":"ContainerStarted","Data":"096486cab878aafc9303a45d996a9baffead66767ca2423843572527af328b00"} Jun 02 19:57:51.846609 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.846565 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-tls\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:51.848880 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:51.848861 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f5800e54-11e6-4369-ab41-1d8454b4caee-node-exporter-tls\") pod \"node-exporter-lqhbt\" (UID: \"f5800e54-11e6-4369-ab41-1d8454b4caee\") " pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:52.074034 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.073927 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lqhbt" Jun 02 19:57:52.083123 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:52.083083 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5800e54_11e6_4369_ab41_1d8454b4caee.slice/crio-fb70b22bcf76a92289cd8d70d4ae3130b75ec1769f5c0b594a6ec1fb8af8a42c WatchSource:0}: Error finding container fb70b22bcf76a92289cd8d70d4ae3130b75ec1769f5c0b594a6ec1fb8af8a42c: Status 404 returned error can't find the container with id fb70b22bcf76a92289cd8d70d4ae3130b75ec1769f5c0b594a6ec1fb8af8a42c Jun 02 19:57:52.193967 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.193923 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:57:52.197254 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.197232 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.202447 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.202376 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 02 19:57:52.202682 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.202665 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vg66d\"" Jun 02 19:57:52.202927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.202911 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 02 19:57:52.203018 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.202995 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 02 19:57:52.203138 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.203088 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 02 19:57:52.203454 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.203438 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 02 19:57:52.203926 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.203909 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 02 19:57:52.204504 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.204426 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 02 19:57:52.204819 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.204800 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 02 19:57:52.205099 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.205080 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 02 19:57:52.222007 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.221949 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:57:52.249334 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249294 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.249334 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249339 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65c75fd0-360c-4d5d-8276-c8dff3956b60-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.249575 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249367 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.249575 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249493 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/65c75fd0-360c-4d5d-8276-c8dff3956b60-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.249575 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249532 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-config-volume\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.249772 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249572 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mvs6\" (UniqueName: \"kubernetes.io/projected/65c75fd0-360c-4d5d-8276-c8dff3956b60-kube-api-access-6mvs6\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.249772 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249605 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-web-config\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.249772 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249634 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65c75fd0-360c-4d5d-8276-c8dff3956b60-config-out\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.249772 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249691 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.249996 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249833 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65c75fd0-360c-4d5d-8276-c8dff3956b60-tls-assets\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.249996 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249855 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.249996 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249880 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c75fd0-360c-4d5d-8276-c8dff3956b60-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.249996 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.249910 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350424 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6mvs6\" (UniqueName: \"kubernetes.io/projected/65c75fd0-360c-4d5d-8276-c8dff3956b60-kube-api-access-6mvs6\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350481 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-web-config\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350511 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65c75fd0-360c-4d5d-8276-c8dff3956b60-config-out\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350566 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350607 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65c75fd0-360c-4d5d-8276-c8dff3956b60-tls-assets\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350631 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350658 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c75fd0-360c-4d5d-8276-c8dff3956b60-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350685 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350762 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350788 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65c75fd0-360c-4d5d-8276-c8dff3956b60-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350815 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350881 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/65c75fd0-360c-4d5d-8276-c8dff3956b60-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.352116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.350909 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-config-volume\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.353584 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.353018 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c75fd0-360c-4d5d-8276-c8dff3956b60-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.353584 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.353276 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/65c75fd0-360c-4d5d-8276-c8dff3956b60-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.353781 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.353689 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65c75fd0-360c-4d5d-8276-c8dff3956b60-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.355702 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.355197 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.355702 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.355496 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-web-config\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.361550 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.361475 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65c75fd0-360c-4d5d-8276-c8dff3956b60-tls-assets\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.361716 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.361637 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.361858 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.361837 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65c75fd0-360c-4d5d-8276-c8dff3956b60-config-out\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.362379 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.362217 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.362379 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.362234 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.363050 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.362989 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-config-volume\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.364703 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.364679 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mvs6\" (UniqueName: \"kubernetes.io/projected/65c75fd0-360c-4d5d-8276-c8dff3956b60-kube-api-access-6mvs6\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.365313 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.365274 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.507731 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.507699 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:57:52.752961 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.752910 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lqhbt" event={"ID":"f5800e54-11e6-4369-ab41-1d8454b4caee","Type":"ContainerStarted","Data":"fb70b22bcf76a92289cd8d70d4ae3130b75ec1769f5c0b594a6ec1fb8af8a42c"} Jun 02 19:57:52.934300 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:52.934243 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:57:53.381258 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:53.381212 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c75fd0_360c_4d5d_8276_c8dff3956b60.slice/crio-e97b1bb359db7ce880c92e41d418998a2b10d853b778adc3ec1548c8b6827991 WatchSource:0}: Error finding container e97b1bb359db7ce880c92e41d418998a2b10d853b778adc3ec1548c8b6827991: Status 404 returned error can't find the container with id e97b1bb359db7ce880c92e41d418998a2b10d853b778adc3ec1548c8b6827991 Jun 02 19:57:53.758971 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:53.758889 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" event={"ID":"62c1ac6c-eef9-4d69-8835-b25a23d64e0c","Type":"ContainerStarted","Data":"7ec2ba7471872404aa1fc2ad5063c98a826f532129ac31b6e9fb72d7fa40d2f7"} Jun 02 19:57:53.758971 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:53.758934 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" event={"ID":"62c1ac6c-eef9-4d69-8835-b25a23d64e0c","Type":"ContainerStarted","Data":"468a670b2d6402ff5a567712c5f9010f1313012a4637e3e68b9e697c9c8f3aae"} Jun 02 19:57:53.758971 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:53.758947 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" event={"ID":"62c1ac6c-eef9-4d69-8835-b25a23d64e0c","Type":"ContainerStarted","Data":"d4014ca6ea1c4459f7148a75833cc3c4a9880c25ea32883a2ef59a5fac9f0819"} Jun 02 19:57:53.761410 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:53.761377 2548 generic.go:358] "Generic (PLEG): container finished" podID="f5800e54-11e6-4369-ab41-1d8454b4caee" containerID="ffc5a6fb78c57500aac3e98484607ee0dceb423215d18ad220df41c81c951631" exitCode=0 Jun 02 19:57:53.761543 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:53.761481 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lqhbt" event={"ID":"f5800e54-11e6-4369-ab41-1d8454b4caee","Type":"ContainerDied","Data":"ffc5a6fb78c57500aac3e98484607ee0dceb423215d18ad220df41c81c951631"} Jun 02 19:57:53.763729 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:53.763703 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" event={"ID":"2115b62d-ce15-4d1b-8fce-7dbee21ace94","Type":"ContainerStarted","Data":"54c49cc62d08f3de4b6602cbc4b64c0d49c405bab4c06c8b66c063a9698d4f1c"} Jun 02 19:57:53.764866 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:53.764835 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerStarted","Data":"e97b1bb359db7ce880c92e41d418998a2b10d853b778adc3ec1548c8b6827991"} Jun 02 19:57:53.780948 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:53.780894 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-g8c8m" podStartSLOduration=1.043427305 podStartE2EDuration="2.780874247s" podCreationTimestamp="2026-06-02 19:57:51 +0000 UTC" firstStartedPulling="2026-06-02 19:57:51.662222971 +0000 UTC m=+158.979436202" lastFinishedPulling="2026-06-02 19:57:53.399669895 +0000 UTC m=+160.716883144" observedRunningTime="2026-06-02 19:57:53.778921122 +0000 UTC m=+161.096134404" watchObservedRunningTime="2026-06-02 19:57:53.780874247 +0000 UTC m=+161.098087502" Jun 02 19:57:53.817535 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:53.817486 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-r2brz" podStartSLOduration=1.771977651 podStartE2EDuration="2.81747007s" podCreationTimestamp="2026-06-02 19:57:51 +0000 UTC" firstStartedPulling="2026-06-02 19:57:51.729824104 +0000 UTC m=+159.047037343" lastFinishedPulling="2026-06-02 19:57:52.775316527 +0000 UTC m=+160.092529762" observedRunningTime="2026-06-02 19:57:53.81623707 +0000 UTC m=+161.133450324" watchObservedRunningTime="2026-06-02 19:57:53.81747007 +0000 UTC m=+161.134683323" Jun 02 19:57:54.769886 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:54.769844 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lqhbt" event={"ID":"f5800e54-11e6-4369-ab41-1d8454b4caee","Type":"ContainerStarted","Data":"459407e134344bed1860d201c3a9465a2723d1862221d7f0a7ca56ad0f887ba4"} Jun 02 19:57:54.769886 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:54.769893 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lqhbt" event={"ID":"f5800e54-11e6-4369-ab41-1d8454b4caee","Type":"ContainerStarted","Data":"8bf88cf1d50e94bea69d1dd08592b25d0b6f569fa0af15af546628d0a4435cd9"} Jun 02 19:57:54.771174 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:54.771148 2548 generic.go:358] "Generic (PLEG): container finished" podID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerID="2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f" exitCode=0 Jun 02 19:57:54.771251 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:54.771231 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerDied","Data":"2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f"} Jun 02 19:57:54.789852 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:54.789799 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lqhbt" podStartSLOduration=2.477301193 podStartE2EDuration="3.789781494s" podCreationTimestamp="2026-06-02 19:57:51 +0000 UTC" firstStartedPulling="2026-06-02 19:57:52.084977925 +0000 UTC m=+159.402191158" lastFinishedPulling="2026-06-02 19:57:53.397458221 +0000 UTC m=+160.714671459" observedRunningTime="2026-06-02 19:57:54.78947711 +0000 UTC m=+162.106690364" watchObservedRunningTime="2026-06-02 19:57:54.789781494 +0000 UTC m=+162.106994749" Jun 02 19:57:55.481975 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.481933 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:57:55.482172 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.481997 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:57:55.482172 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.482067 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:57:55.485003 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.484973 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7952dddc-f51b-4a0c-8a23-feecfec0457a-metrics-tls\") pod \"dns-default-rmbfk\" (UID: \"7952dddc-f51b-4a0c-8a23-feecfec0457a\") " pod="openshift-dns/dns-default-rmbfk" Jun 02 19:57:55.485139 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.485099 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") pod \"image-registry-6884588f7d-rvb4x\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:57:55.485139 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.485110 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bade01b-4276-40ba-b81a-d607f88064e2-cert\") pod \"ingress-canary-jjfnh\" (UID: \"3bade01b-4276-40ba-b81a-d607f88064e2\") " pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:57:55.546015 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.545983 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vtp5s\"" Jun 02 19:57:55.546921 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.546903 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mg65m\"" Jun 02 19:57:55.553793 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.553771 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rmbfk" Jun 02 19:57:55.553920 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.553868 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:57:55.617316 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.616871 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-548f54f5f7-hqrg8"] Jun 02 19:57:55.621193 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.620663 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.624937 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.624409 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 02 19:57:55.624937 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.624615 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-d9mvd\"" Jun 02 19:57:55.624937 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.624674 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bdstq88tgnr2m\"" Jun 02 19:57:55.625197 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.625097 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 02 19:57:55.625197 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.625188 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 02 19:57:55.625717 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.625560 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 02 19:57:55.636223 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.635607 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-548f54f5f7-hqrg8"] Jun 02 19:57:55.684004 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.683791 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/32785a32-52d3-4401-ab3c-915abb3835e7-secret-metrics-server-client-certs\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.684004 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.683842 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/32785a32-52d3-4401-ab3c-915abb3835e7-secret-metrics-server-tls\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.684004 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.683872 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32785a32-52d3-4401-ab3c-915abb3835e7-client-ca-bundle\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.684004 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.683901 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6w7r\" (UniqueName: \"kubernetes.io/projected/32785a32-52d3-4401-ab3c-915abb3835e7-kube-api-access-g6w7r\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.684004 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.683940 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/32785a32-52d3-4401-ab3c-915abb3835e7-audit-log\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.684381 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.684064 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/32785a32-52d3-4401-ab3c-915abb3835e7-metrics-server-audit-profiles\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.684381 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.684112 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32785a32-52d3-4401-ab3c-915abb3835e7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.728116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.728077 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6884588f7d-rvb4x"] Jun 02 19:57:55.733442 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:55.733368 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58c985a_62d6_4fa6_85f2_a79d3cebac80.slice/crio-4c1264fe567f9b40bdf9321550e66165aa0835f9a81601265c06f1f88b1c179e WatchSource:0}: Error finding container 4c1264fe567f9b40bdf9321550e66165aa0835f9a81601265c06f1f88b1c179e: Status 404 returned error can't find the container with id 4c1264fe567f9b40bdf9321550e66165aa0835f9a81601265c06f1f88b1c179e Jun 02 19:57:55.742979 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.742951 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rmbfk"] Jun 02 19:57:55.746126 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.746101 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697"] Jun 02 19:57:55.746836 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:55.746795 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7952dddc_f51b_4a0c_8a23_feecfec0457a.slice/crio-282db2633487a45906bb5450e4bcf7b9c9b8855cb73750e2f42a25ecb708a252 WatchSource:0}: Error finding container 282db2633487a45906bb5450e4bcf7b9c9b8855cb73750e2f42a25ecb708a252: Status 404 returned error can't find the container with id 282db2633487a45906bb5450e4bcf7b9c9b8855cb73750e2f42a25ecb708a252 Jun 02 19:57:55.749308 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.749255 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697" Jun 02 19:57:55.751756 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.751717 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-c26g4\"" Jun 02 19:57:55.752144 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.752008 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 02 19:57:55.758134 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.758111 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697"] Jun 02 19:57:55.776309 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.776267 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" event={"ID":"d58c985a-62d6-4fa6-85f2-a79d3cebac80","Type":"ContainerStarted","Data":"4c1264fe567f9b40bdf9321550e66165aa0835f9a81601265c06f1f88b1c179e"} Jun 02 19:57:55.777418 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.777376 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rmbfk" event={"ID":"7952dddc-f51b-4a0c-8a23-feecfec0457a","Type":"ContainerStarted","Data":"282db2633487a45906bb5450e4bcf7b9c9b8855cb73750e2f42a25ecb708a252"} Jun 02 19:57:55.785016 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.784990 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/32785a32-52d3-4401-ab3c-915abb3835e7-secret-metrics-server-client-certs\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.785146 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.785026 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/32785a32-52d3-4401-ab3c-915abb3835e7-secret-metrics-server-tls\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.785146 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.785046 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32785a32-52d3-4401-ab3c-915abb3835e7-client-ca-bundle\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.785146 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.785071 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g6w7r\" (UniqueName: \"kubernetes.io/projected/32785a32-52d3-4401-ab3c-915abb3835e7-kube-api-access-g6w7r\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.785146 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.785106 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/32785a32-52d3-4401-ab3c-915abb3835e7-audit-log\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.785303 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.785145 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/32785a32-52d3-4401-ab3c-915abb3835e7-metrics-server-audit-profiles\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.785303 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.785178 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32785a32-52d3-4401-ab3c-915abb3835e7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.785967 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.785933 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32785a32-52d3-4401-ab3c-915abb3835e7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.786245 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.786216 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/32785a32-52d3-4401-ab3c-915abb3835e7-audit-log\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.787116 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.787055 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/32785a32-52d3-4401-ab3c-915abb3835e7-metrics-server-audit-profiles\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.788980 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.788960 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/32785a32-52d3-4401-ab3c-915abb3835e7-secret-metrics-server-client-certs\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.789817 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.789719 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/32785a32-52d3-4401-ab3c-915abb3835e7-secret-metrics-server-tls\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.789999 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.789939 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32785a32-52d3-4401-ab3c-915abb3835e7-client-ca-bundle\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.793965 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.793922 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6w7r\" (UniqueName: \"kubernetes.io/projected/32785a32-52d3-4401-ab3c-915abb3835e7-kube-api-access-g6w7r\") pod \"metrics-server-548f54f5f7-hqrg8\" (UID: \"32785a32-52d3-4401-ab3c-915abb3835e7\") " pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.886353 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.886315 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6cfc3c1d-ea45-4b98-a84b-1ee69bf9dc39-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-7f697\" (UID: \"6cfc3c1d-ea45-4b98-a84b-1ee69bf9dc39\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697" Jun 02 19:57:55.942207 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.942171 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:57:55.987315 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.987212 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6cfc3c1d-ea45-4b98-a84b-1ee69bf9dc39-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-7f697\" (UID: \"6cfc3c1d-ea45-4b98-a84b-1ee69bf9dc39\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697" Jun 02 19:57:55.990247 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:55.990207 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6cfc3c1d-ea45-4b98-a84b-1ee69bf9dc39-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-7f697\" (UID: \"6cfc3c1d-ea45-4b98-a84b-1ee69bf9dc39\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697" Jun 02 19:57:56.062269 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.062214 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697" Jun 02 19:57:56.366699 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.366666 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697"] Jun 02 19:57:56.393066 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.392021 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-548f54f5f7-hqrg8"] Jun 02 19:57:56.395615 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:56.395584 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32785a32_52d3_4401_ab3c_915abb3835e7.slice/crio-aeb3d320a3551b285b8b9bb240c377af2125d159847540fe8ed67d39408617dd WatchSource:0}: Error finding container aeb3d320a3551b285b8b9bb240c377af2125d159847540fe8ed67d39408617dd: Status 404 returned error can't find the container with id aeb3d320a3551b285b8b9bb240c377af2125d159847540fe8ed67d39408617dd Jun 02 19:57:56.783751 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.783654 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerStarted","Data":"53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368"} Jun 02 19:57:56.783751 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.783695 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerStarted","Data":"2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e"} Jun 02 19:57:56.783751 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.783708 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerStarted","Data":"6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d"} Jun 02 19:57:56.783751 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.783721 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerStarted","Data":"5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e"} Jun 02 19:57:56.784259 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.783734 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerStarted","Data":"53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd"} Jun 02 19:57:56.784784 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.784755 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" event={"ID":"32785a32-52d3-4401-ab3c-915abb3835e7","Type":"ContainerStarted","Data":"aeb3d320a3551b285b8b9bb240c377af2125d159847540fe8ed67d39408617dd"} Jun 02 19:57:56.786006 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.785985 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" event={"ID":"d58c985a-62d6-4fa6-85f2-a79d3cebac80","Type":"ContainerStarted","Data":"dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478"} Jun 02 19:57:56.786155 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.786130 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:57:56.787030 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.787007 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697" event={"ID":"6cfc3c1d-ea45-4b98-a84b-1ee69bf9dc39","Type":"ContainerStarted","Data":"97a52a734b2d7318507f9e180207304adfa3236e9283967f8e708490d7ce73b0"} Jun 02 19:57:56.808169 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:56.808120 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" podStartSLOduration=162.808105888 podStartE2EDuration="2m42.808105888s" podCreationTimestamp="2026-06-02 19:55:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:57:56.807656714 +0000 UTC m=+164.124869968" watchObservedRunningTime="2026-06-02 19:57:56.808105888 +0000 UTC m=+164.125319142" Jun 02 19:57:57.307391 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.307342 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:57:57.311122 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.311097 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.314556 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.314301 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 02 19:57:57.314556 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.314390 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 02 19:57:57.314556 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.314309 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pvcrv\"" Jun 02 19:57:57.316640 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.316617 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 02 19:57:57.317224 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.317004 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 02 19:57:57.317224 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.317029 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 02 19:57:57.317224 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.317034 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 02 19:57:57.317224 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.317114 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 02 19:57:57.317480 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.317317 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 02 19:57:57.317480 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.317387 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 02 19:57:57.317916 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.317723 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fvknc94923rfs\"" Jun 02 19:57:57.319189 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.318091 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 02 19:57:57.320304 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.320278 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 02 19:57:57.323398 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.323377 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 02 19:57:57.332451 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.332419 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:57:57.404065 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.403987 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404065 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404046 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-config\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404306 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404083 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-config-out\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404306 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404112 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404306 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404189 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404306 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404219 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404306 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404261 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404306 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404283 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404306 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404306 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404661 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404345 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404661 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404373 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404661 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404459 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404661 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404608 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404661 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404634 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-web-config\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404858 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404712 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404858 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404772 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404858 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404799 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.404858 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.404822 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt6rq\" (UniqueName: \"kubernetes.io/projected/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-kube-api-access-tt6rq\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506206 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506169 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506206 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506222 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506442 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506259 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506442 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506291 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506442 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506314 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-web-config\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506442 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506347 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506442 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506386 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506442 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506421 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506727 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506449 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tt6rq\" (UniqueName: \"kubernetes.io/projected/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-kube-api-access-tt6rq\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506727 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506482 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506727 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506524 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-config\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506727 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506560 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-config-out\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506727 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506586 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506727 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506627 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506727 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506655 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506727 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506699 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.506727 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506723 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.507167 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.506770 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.507217 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.507161 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.507264 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.507239 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.510766 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.510719 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.511605 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.511582 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.512653 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.512626 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.512793 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.512643 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.513192 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.513121 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-config\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.513693 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.513674 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.514927 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.514884 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.516045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.515649 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.516045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.515862 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-config-out\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.516045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.515968 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.516045 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.516011 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-web-config\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.516282 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.516102 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.517658 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.517633 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.517658 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.517646 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.517835 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.517663 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.520121 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.520080 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt6rq\" (UniqueName: \"kubernetes.io/projected/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-kube-api-access-tt6rq\") pod \"prometheus-k8s-0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:57.629625 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:57.629013 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:57:58.853416 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:58.853391 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:57:58.867760 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:57:58.867707 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44082f80_c134_4f39_a9ab_1a9f3f7dc9d0.slice/crio-01159e8e8a9b292f78aac7db815f12629ab836edc522c7590fdfe680492e04d7 WatchSource:0}: Error finding container 01159e8e8a9b292f78aac7db815f12629ab836edc522c7590fdfe680492e04d7: Status 404 returned error can't find the container with id 01159e8e8a9b292f78aac7db815f12629ab836edc522c7590fdfe680492e04d7 Jun 02 19:57:59.805543 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.805505 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerStarted","Data":"6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18"} Jun 02 19:57:59.807223 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.807194 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rmbfk" event={"ID":"7952dddc-f51b-4a0c-8a23-feecfec0457a","Type":"ContainerStarted","Data":"bf491f27e87f3f4f2468cc0c9dd4d86bb6eab9cad1abee4a7f21200a36bf2f22"} Jun 02 19:57:59.807223 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.807225 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rmbfk" event={"ID":"7952dddc-f51b-4a0c-8a23-feecfec0457a","Type":"ContainerStarted","Data":"e15e73346ab3923cc48c0f2b09ce49c343b0fb75eb8c73eca26026f4e7d2ad4b"} Jun 02 19:57:59.807425 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.807322 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rmbfk" Jun 02 19:57:59.808587 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.808565 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" event={"ID":"32785a32-52d3-4401-ab3c-915abb3835e7","Type":"ContainerStarted","Data":"ca3c7bce54b69ab046d62d2c4002feea87f923784bffd933c1c2087ef31deaa7"} Jun 02 19:57:59.809784 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.809761 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697" event={"ID":"6cfc3c1d-ea45-4b98-a84b-1ee69bf9dc39","Type":"ContainerStarted","Data":"4c65b4cc2187fcfeee11c9599ace7ebd27007bd05c665aad70d590c51f5b4907"} Jun 02 19:57:59.809864 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.809792 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697" Jun 02 19:57:59.811168 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.811141 2548 generic.go:358] "Generic (PLEG): container finished" podID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerID="a9c73e4f5ce324961f4bd132b8b5a1dfcce5cd1755b11249c988149229d2d441" exitCode=0 Jun 02 19:57:59.811274 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.811170 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerDied","Data":"a9c73e4f5ce324961f4bd132b8b5a1dfcce5cd1755b11249c988149229d2d441"} Jun 02 19:57:59.811274 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.811202 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerStarted","Data":"01159e8e8a9b292f78aac7db815f12629ab836edc522c7590fdfe680492e04d7"} Jun 02 19:57:59.815423 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.815399 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697" Jun 02 19:57:59.836853 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.836797 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.351025061 podStartE2EDuration="7.836781545s" podCreationTimestamp="2026-06-02 19:57:52 +0000 UTC" firstStartedPulling="2026-06-02 19:57:53.395691375 +0000 UTC m=+160.712904607" lastFinishedPulling="2026-06-02 19:57:58.881447855 +0000 UTC m=+166.198661091" observedRunningTime="2026-06-02 19:57:59.835213486 +0000 UTC m=+167.152426740" watchObservedRunningTime="2026-06-02 19:57:59.836781545 +0000 UTC m=+167.153994799" Jun 02 19:57:59.856319 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.856261 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-7f697" podStartSLOduration=2.387322423 podStartE2EDuration="4.856242434s" podCreationTimestamp="2026-06-02 19:57:55 +0000 UTC" firstStartedPulling="2026-06-02 19:57:56.375214661 +0000 UTC m=+163.692427907" lastFinishedPulling="2026-06-02 19:57:58.844134675 +0000 UTC m=+166.161347918" observedRunningTime="2026-06-02 19:57:59.85407324 +0000 UTC m=+167.171286511" watchObservedRunningTime="2026-06-02 19:57:59.856242434 +0000 UTC m=+167.173455690" Jun 02 19:57:59.905699 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.905649 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-rmbfk" podStartSLOduration=129.812193135 podStartE2EDuration="2m12.90563412s" podCreationTimestamp="2026-06-02 19:55:47 +0000 UTC" firstStartedPulling="2026-06-02 19:57:55.749381041 +0000 UTC m=+163.066594280" lastFinishedPulling="2026-06-02 19:57:58.842822033 +0000 UTC m=+166.160035265" observedRunningTime="2026-06-02 19:57:59.904322544 +0000 UTC m=+167.221535798" watchObservedRunningTime="2026-06-02 19:57:59.90563412 +0000 UTC m=+167.222847386" Jun 02 19:57:59.923206 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:57:59.923138 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" podStartSLOduration=2.476838636 podStartE2EDuration="4.923119564s" podCreationTimestamp="2026-06-02 19:57:55 +0000 UTC" firstStartedPulling="2026-06-02 19:57:56.397656271 +0000 UTC m=+163.714869503" lastFinishedPulling="2026-06-02 19:57:58.843937189 +0000 UTC m=+166.161150431" observedRunningTime="2026-06-02 19:57:59.922469699 +0000 UTC m=+167.239682954" watchObservedRunningTime="2026-06-02 19:57:59.923119564 +0000 UTC m=+167.240332821" Jun 02 19:58:03.825401 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:03.825365 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerStarted","Data":"45d11b74a95bcefcdb091cfedd78a8ee668443fe396d0aa90bf89966309e368b"} Jun 02 19:58:03.825817 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:03.825410 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerStarted","Data":"bb126eea9bf415595c74c96e4af3b713ba37321b79ddc95cef1bee478f960999"} Jun 02 19:58:05.313261 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:05.313152 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:58:05.313261 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:05.313185 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:58:05.316310 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:05.316279 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-l7ztv\"" Jun 02 19:58:05.324464 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:05.324431 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jjfnh" Jun 02 19:58:05.485428 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:05.485390 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jjfnh"] Jun 02 19:58:05.489562 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:58:05.489526 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bade01b_4276_40ba_b81a_d607f88064e2.slice/crio-f6fe837d2b5f14cab3b1eee4eb354a906237ae391cb4d690f439385f1aa4cc42 WatchSource:0}: Error finding container f6fe837d2b5f14cab3b1eee4eb354a906237ae391cb4d690f439385f1aa4cc42: Status 404 returned error can't find the container with id f6fe837d2b5f14cab3b1eee4eb354a906237ae391cb4d690f439385f1aa4cc42 Jun 02 19:58:05.833395 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:05.833356 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jjfnh" event={"ID":"3bade01b-4276-40ba-b81a-d607f88064e2","Type":"ContainerStarted","Data":"f6fe837d2b5f14cab3b1eee4eb354a906237ae391cb4d690f439385f1aa4cc42"} Jun 02 19:58:05.836846 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:05.836812 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerStarted","Data":"2aa396d95d0d7a2ad0f35accc25be2796c90ce25da34179e8b1dd82c2d5f77d3"} Jun 02 19:58:05.836846 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:05.836847 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerStarted","Data":"d86dc665b6ae7441ac99331764fc128548de6ce893b9f37208f11afdf8a8f5db"} Jun 02 19:58:05.837042 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:05.836861 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerStarted","Data":"564407e3a5bf3cfd68cf4886b2c38138be883ee38244bdfb547ebf63029b0f43"} Jun 02 19:58:05.837042 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:05.836873 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerStarted","Data":"3f1e1cee45bbfef5b4bfc7d74a1b67a53b50b0fe894b0631a2dfd63de1d00981"} Jun 02 19:58:05.867709 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:05.867651 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.904633215 podStartE2EDuration="8.867630224s" podCreationTimestamp="2026-06-02 19:57:57 +0000 UTC" firstStartedPulling="2026-06-02 19:57:59.812479599 +0000 UTC m=+167.129692832" lastFinishedPulling="2026-06-02 19:58:04.77547661 +0000 UTC m=+172.092689841" observedRunningTime="2026-06-02 19:58:05.865237388 +0000 UTC m=+173.182450642" watchObservedRunningTime="2026-06-02 19:58:05.867630224 +0000 UTC m=+173.184843481" Jun 02 19:58:07.629913 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:07.629870 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:58:07.844466 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:07.844424 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jjfnh" event={"ID":"3bade01b-4276-40ba-b81a-d607f88064e2","Type":"ContainerStarted","Data":"92db1b893fc029f558231652ae15908e6b4b53c64da5827d2d846c1d379cb2f8"} Jun 02 19:58:07.862722 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:07.862673 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jjfnh" podStartSLOduration=139.399335508 podStartE2EDuration="2m20.862658324s" podCreationTimestamp="2026-06-02 19:55:47 +0000 UTC" firstStartedPulling="2026-06-02 19:58:05.491997705 +0000 UTC m=+172.809210953" lastFinishedPulling="2026-06-02 19:58:06.955320533 +0000 UTC m=+174.272533769" observedRunningTime="2026-06-02 19:58:07.861400092 +0000 UTC m=+175.178613349" watchObservedRunningTime="2026-06-02 19:58:07.862658324 +0000 UTC m=+175.179871577" Jun 02 19:58:09.816422 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:09.816393 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rmbfk" Jun 02 19:58:15.532801 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:15.532760 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6884588f7d-rvb4x"] Jun 02 19:58:15.537091 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:15.537059 2548 patch_prober.go:28] interesting pod/image-registry-6884588f7d-rvb4x container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 02 19:58:15.537241 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:15.537116 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" podUID="d58c985a-62d6-4fa6-85f2-a79d3cebac80" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 19:58:15.943234 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:15.943197 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:58:15.943413 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:15.943266 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:58:17.875529 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:17.875488 2548 generic.go:358] "Generic (PLEG): container finished" podID="1493c956-abd3-4deb-be9c-9ac0eb2eb886" containerID="3072b4bcbd57f98d281380fbf4848efdfdc11fb08173b57127b1fe8bd719db7d" exitCode=0 Jun 02 19:58:17.875940 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:17.875567 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-kgq9t" event={"ID":"1493c956-abd3-4deb-be9c-9ac0eb2eb886","Type":"ContainerDied","Data":"3072b4bcbd57f98d281380fbf4848efdfdc11fb08173b57127b1fe8bd719db7d"} Jun 02 19:58:17.876005 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:17.875988 2548 scope.go:117] "RemoveContainer" containerID="3072b4bcbd57f98d281380fbf4848efdfdc11fb08173b57127b1fe8bd719db7d" Jun 02 19:58:18.880784 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:18.880730 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-kgq9t" event={"ID":"1493c956-abd3-4deb-be9c-9ac0eb2eb886","Type":"ContainerStarted","Data":"116e020326e87a9a619f3c38e31387d0b287924bb49e8dc10a5eff4d3e3566d4"} Jun 02 19:58:19.632293 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:19.632269 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rmbfk_7952dddc-f51b-4a0c-8a23-feecfec0457a/dns/0.log" Jun 02 19:58:19.831701 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:19.831670 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rmbfk_7952dddc-f51b-4a0c-8a23-feecfec0457a/kube-rbac-proxy/0.log" Jun 02 19:58:20.231619 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:20.231591 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lz2mg_c3674c37-2e21-4799-9545-d69c1985b721/dns-node-resolver/0.log" Jun 02 19:58:21.231601 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:21.231568 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jjfnh_3bade01b-4276-40ba-b81a-d607f88064e2/serve-healthcheck-canary/0.log" Jun 02 19:58:25.536821 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:25.536793 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:58:35.948052 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:35.948021 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:58:35.952165 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:35.952147 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-548f54f5f7-hqrg8" Jun 02 19:58:37.946883 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:37.946848 2548 generic.go:358] "Generic (PLEG): container finished" podID="5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f" containerID="742f5d0a2dac9584701d83515e20366f83f74dce7b12ea3e93ab1d6724f3855e" exitCode=0 Jun 02 19:58:37.947268 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:37.946924 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" event={"ID":"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f","Type":"ContainerDied","Data":"742f5d0a2dac9584701d83515e20366f83f74dce7b12ea3e93ab1d6724f3855e"} Jun 02 19:58:37.947318 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:37.947291 2548 scope.go:117] "RemoveContainer" containerID="742f5d0a2dac9584701d83515e20366f83f74dce7b12ea3e93ab1d6724f3855e" Jun 02 19:58:38.951944 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:38.951904 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7hmx9" event={"ID":"5ab1f9fc-c1b0-419a-a8c9-ebc565a5e35f","Type":"ContainerStarted","Data":"8a59f058b738e1365a287e85d4f81d62d96a9ae62880d100153d08c9fd672342"} Jun 02 19:58:38.953127 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:38.953102 2548 generic.go:358] "Generic (PLEG): container finished" podID="76e3be1c-425f-407d-8805-bc27d505f922" containerID="179f38cde7bba363bbb124ad7906f2ab6faebae6f01e64d3ceec4f104d70ea2e" exitCode=0 Jun 02 19:58:38.953218 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:38.953168 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" event={"ID":"76e3be1c-425f-407d-8805-bc27d505f922","Type":"ContainerDied","Data":"179f38cde7bba363bbb124ad7906f2ab6faebae6f01e64d3ceec4f104d70ea2e"} Jun 02 19:58:38.953437 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:38.953422 2548 scope.go:117] "RemoveContainer" containerID="179f38cde7bba363bbb124ad7906f2ab6faebae6f01e64d3ceec4f104d70ea2e" Jun 02 19:58:39.958475 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:39.958439 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-khsh7" event={"ID":"76e3be1c-425f-407d-8805-bc27d505f922","Type":"ContainerStarted","Data":"352cc46ed7cf14cc847352d3170035b9d9890f5668459a8c21072fcf53cc8766"} Jun 02 19:58:40.552934 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.552871 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" podUID="d58c985a-62d6-4fa6-85f2-a79d3cebac80" containerName="registry" containerID="cri-o://dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478" gracePeriod=30 Jun 02 19:58:40.805933 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.805869 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:58:40.931529 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.931487 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d58c985a-62d6-4fa6-85f2-a79d3cebac80-installation-pull-secrets\") pod \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " Jun 02 19:58:40.931731 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.931541 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45h55\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-kube-api-access-45h55\") pod \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " Jun 02 19:58:40.931731 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.931569 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d58c985a-62d6-4fa6-85f2-a79d3cebac80-trusted-ca\") pod \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " Jun 02 19:58:40.931731 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.931620 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") pod \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " Jun 02 19:58:40.931731 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.931643 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-bound-sa-token\") pod \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " Jun 02 19:58:40.931993 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.931803 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-certificates\") pod \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " Jun 02 19:58:40.931993 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.931857 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d58c985a-62d6-4fa6-85f2-a79d3cebac80-ca-trust-extracted\") pod \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " Jun 02 19:58:40.931993 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.931945 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d58c985a-62d6-4fa6-85f2-a79d3cebac80-image-registry-private-configuration\") pod \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\" (UID: \"d58c985a-62d6-4fa6-85f2-a79d3cebac80\") " Jun 02 19:58:40.932171 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.932145 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58c985a-62d6-4fa6-85f2-a79d3cebac80-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d58c985a-62d6-4fa6-85f2-a79d3cebac80" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:58:40.932261 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.932229 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d58c985a-62d6-4fa6-85f2-a79d3cebac80" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:58:40.932336 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.932315 2548 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d58c985a-62d6-4fa6-85f2-a79d3cebac80-trusted-ca\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:58:40.932393 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.932343 2548 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-certificates\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:58:40.934198 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.934144 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58c985a-62d6-4fa6-85f2-a79d3cebac80-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d58c985a-62d6-4fa6-85f2-a79d3cebac80" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:58:40.934564 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.934532 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d58c985a-62d6-4fa6-85f2-a79d3cebac80" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:58:40.934564 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.934542 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58c985a-62d6-4fa6-85f2-a79d3cebac80-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d58c985a-62d6-4fa6-85f2-a79d3cebac80" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:58:40.934722 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.934597 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d58c985a-62d6-4fa6-85f2-a79d3cebac80" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:58:40.934807 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.934757 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-kube-api-access-45h55" (OuterVolumeSpecName: "kube-api-access-45h55") pod "d58c985a-62d6-4fa6-85f2-a79d3cebac80" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80"). InnerVolumeSpecName "kube-api-access-45h55". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:58:40.939854 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.939828 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58c985a-62d6-4fa6-85f2-a79d3cebac80-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d58c985a-62d6-4fa6-85f2-a79d3cebac80" (UID: "d58c985a-62d6-4fa6-85f2-a79d3cebac80"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:58:40.963051 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.963019 2548 generic.go:358] "Generic (PLEG): container finished" podID="d58c985a-62d6-4fa6-85f2-a79d3cebac80" containerID="dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478" exitCode=0 Jun 02 19:58:40.963432 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.963078 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" Jun 02 19:58:40.963432 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.963108 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" event={"ID":"d58c985a-62d6-4fa6-85f2-a79d3cebac80","Type":"ContainerDied","Data":"dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478"} Jun 02 19:58:40.963432 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.963149 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6884588f7d-rvb4x" event={"ID":"d58c985a-62d6-4fa6-85f2-a79d3cebac80","Type":"ContainerDied","Data":"4c1264fe567f9b40bdf9321550e66165aa0835f9a81601265c06f1f88b1c179e"} Jun 02 19:58:40.963432 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.963165 2548 scope.go:117] "RemoveContainer" containerID="dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478" Jun 02 19:58:40.972301 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.972278 2548 scope.go:117] "RemoveContainer" containerID="dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478" Jun 02 19:58:40.972587 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:58:40.972568 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478\": container with ID starting with dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478 not found: ID does not exist" containerID="dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478" Jun 02 19:58:40.972630 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.972597 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478"} err="failed to get container status \"dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478\": rpc error: code = NotFound desc = could not find container \"dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478\": container with ID starting with dc30875770a25edf732ef1fdc030e41b5c1d82769c923c5ccb487ffb54205478 not found: ID does not exist" Jun 02 19:58:40.985215 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.985185 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6884588f7d-rvb4x"] Jun 02 19:58:40.992530 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:40.992502 2548 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6884588f7d-rvb4x"] Jun 02 19:58:41.033340 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:41.033301 2548 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d58c985a-62d6-4fa6-85f2-a79d3cebac80-ca-trust-extracted\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:58:41.033340 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:41.033334 2548 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d58c985a-62d6-4fa6-85f2-a79d3cebac80-image-registry-private-configuration\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:58:41.033340 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:41.033345 2548 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d58c985a-62d6-4fa6-85f2-a79d3cebac80-installation-pull-secrets\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:58:41.033574 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:41.033355 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45h55\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-kube-api-access-45h55\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:58:41.033574 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:41.033364 2548 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-registry-tls\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:58:41.033574 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:41.033373 2548 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d58c985a-62d6-4fa6-85f2-a79d3cebac80-bound-sa-token\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:58:41.317667 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:41.317635 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d58c985a-62d6-4fa6-85f2-a79d3cebac80" path="/var/lib/kubelet/pods/d58c985a-62d6-4fa6-85f2-a79d3cebac80/volumes" Jun 02 19:58:57.630632 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:57.630574 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:58:57.650534 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:57.650501 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:58:58.028438 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:58:58.028409 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:11.514289 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:11.514254 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:59:11.514696 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:11.514658 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="alertmanager" containerID="cri-o://53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd" gracePeriod=120 Jun 02 19:59:11.514800 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:11.514726 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="kube-rbac-proxy-metric" containerID="cri-o://53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368" gracePeriod=120 Jun 02 19:59:11.514869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:11.514771 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="kube-rbac-proxy-web" containerID="cri-o://6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d" gracePeriod=120 Jun 02 19:59:11.514869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:11.514802 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="config-reloader" containerID="cri-o://5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e" gracePeriod=120 Jun 02 19:59:11.514869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:11.514817 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="kube-rbac-proxy" containerID="cri-o://2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e" gracePeriod=120 Jun 02 19:59:11.515026 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:11.514827 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="prom-label-proxy" containerID="cri-o://6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18" gracePeriod=120 Jun 02 19:59:12.057275 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.057243 2548 generic.go:358] "Generic (PLEG): container finished" podID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerID="6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18" exitCode=0 Jun 02 19:59:12.057275 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.057269 2548 generic.go:358] "Generic (PLEG): container finished" podID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerID="2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e" exitCode=0 Jun 02 19:59:12.057275 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.057277 2548 generic.go:358] "Generic (PLEG): container finished" podID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerID="5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e" exitCode=0 Jun 02 19:59:12.057275 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.057284 2548 generic.go:358] "Generic (PLEG): container finished" podID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerID="53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd" exitCode=0 Jun 02 19:59:12.057518 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.057310 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerDied","Data":"6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18"} Jun 02 19:59:12.057518 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.057345 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerDied","Data":"2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e"} Jun 02 19:59:12.057518 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.057356 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerDied","Data":"5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e"} Jun 02 19:59:12.057518 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.057364 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerDied","Data":"53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd"} Jun 02 19:59:12.779346 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.779321 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:12.927078 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927020 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c75fd0-360c-4d5d-8276-c8dff3956b60-alertmanager-trusted-ca-bundle\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927286 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927097 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-cluster-tls-config\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927286 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927125 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65c75fd0-360c-4d5d-8276-c8dff3956b60-config-out\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927286 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927211 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-main-tls\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927286 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927262 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy-web\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927516 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927297 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-web-config\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927516 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927328 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65c75fd0-360c-4d5d-8276-c8dff3956b60-metrics-client-ca\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927516 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927370 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-config-volume\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927516 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927399 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mvs6\" (UniqueName: \"kubernetes.io/projected/65c75fd0-360c-4d5d-8276-c8dff3956b60-kube-api-access-6mvs6\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927516 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927426 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy-metric\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927516 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927453 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65c75fd0-360c-4d5d-8276-c8dff3956b60-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:59:12.927516 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927510 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927897 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927545 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65c75fd0-360c-4d5d-8276-c8dff3956b60-tls-assets\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927897 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927601 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/65c75fd0-360c-4d5d-8276-c8dff3956b60-alertmanager-main-db\") pod \"65c75fd0-360c-4d5d-8276-c8dff3956b60\" (UID: \"65c75fd0-360c-4d5d-8276-c8dff3956b60\") " Jun 02 19:59:12.927998 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.927933 2548 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c75fd0-360c-4d5d-8276-c8dff3956b60-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:12.928080 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.928053 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65c75fd0-360c-4d5d-8276-c8dff3956b60-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:59:12.928268 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.928230 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c75fd0-360c-4d5d-8276-c8dff3956b60-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:59:12.930188 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.930160 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:12.930377 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.930339 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c75fd0-360c-4d5d-8276-c8dff3956b60-config-out" (OuterVolumeSpecName: "config-out") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:59:12.930599 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.930573 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:12.930990 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.930960 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c75fd0-360c-4d5d-8276-c8dff3956b60-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:59:12.931580 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.931328 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c75fd0-360c-4d5d-8276-c8dff3956b60-kube-api-access-6mvs6" (OuterVolumeSpecName: "kube-api-access-6mvs6") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "kube-api-access-6mvs6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:59:12.931580 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.931556 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:12.931720 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.931694 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:12.931992 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.931972 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-config-volume" (OuterVolumeSpecName: "config-volume") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:12.934885 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.934857 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:12.942158 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:12.942131 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-web-config" (OuterVolumeSpecName: "web-config") pod "65c75fd0-360c-4d5d-8276-c8dff3956b60" (UID: "65c75fd0-360c-4d5d-8276-c8dff3956b60"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:13.028602 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.028562 2548 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65c75fd0-360c-4d5d-8276-c8dff3956b60-metrics-client-ca\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:13.028602 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.028595 2548 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-config-volume\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:13.028602 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.028607 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mvs6\" (UniqueName: \"kubernetes.io/projected/65c75fd0-360c-4d5d-8276-c8dff3956b60-kube-api-access-6mvs6\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:13.028869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.028617 2548 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:13.028869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.028627 2548 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:13.028869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.028636 2548 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65c75fd0-360c-4d5d-8276-c8dff3956b60-tls-assets\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:13.028869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.028644 2548 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/65c75fd0-360c-4d5d-8276-c8dff3956b60-alertmanager-main-db\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:13.028869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.028653 2548 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-cluster-tls-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:13.028869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.028662 2548 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65c75fd0-360c-4d5d-8276-c8dff3956b60-config-out\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:13.028869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.028671 2548 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-main-tls\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:13.028869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.028679 2548 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:13.028869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.028688 2548 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65c75fd0-360c-4d5d-8276-c8dff3956b60-web-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:13.067352 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.067314 2548 generic.go:358] "Generic (PLEG): container finished" podID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerID="53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368" exitCode=0 Jun 02 19:59:13.067352 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.067347 2548 generic.go:358] "Generic (PLEG): container finished" podID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerID="6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d" exitCode=0 Jun 02 19:59:13.067549 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.067406 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerDied","Data":"53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368"} Jun 02 19:59:13.067549 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.067452 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerDied","Data":"6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d"} Jun 02 19:59:13.067549 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.067456 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.067549 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.067467 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65c75fd0-360c-4d5d-8276-c8dff3956b60","Type":"ContainerDied","Data":"e97b1bb359db7ce880c92e41d418998a2b10d853b778adc3ec1548c8b6827991"} Jun 02 19:59:13.067549 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.067486 2548 scope.go:117] "RemoveContainer" containerID="6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18" Jun 02 19:59:13.075887 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.075864 2548 scope.go:117] "RemoveContainer" containerID="53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368" Jun 02 19:59:13.083199 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.083168 2548 scope.go:117] "RemoveContainer" containerID="2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e" Jun 02 19:59:13.090320 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.090292 2548 scope.go:117] "RemoveContainer" containerID="6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d" Jun 02 19:59:13.097521 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.097496 2548 scope.go:117] "RemoveContainer" containerID="5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e" Jun 02 19:59:13.101765 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.101711 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:59:13.105606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.105586 2548 scope.go:117] "RemoveContainer" containerID="53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd" Jun 02 19:59:13.110074 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.110046 2548 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:59:13.113782 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.113722 2548 scope.go:117] "RemoveContainer" containerID="2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f" Jun 02 19:59:13.121793 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.121763 2548 scope.go:117] "RemoveContainer" containerID="6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18" Jun 02 19:59:13.122142 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:59:13.122112 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18\": container with ID starting with 6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18 not found: ID does not exist" containerID="6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18" Jun 02 19:59:13.122248 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.122155 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18"} err="failed to get container status \"6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18\": rpc error: code = NotFound desc = could not find container \"6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18\": container with ID starting with 6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18 not found: ID does not exist" Jun 02 19:59:13.122248 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.122184 2548 scope.go:117] "RemoveContainer" containerID="53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368" Jun 02 19:59:13.122461 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:59:13.122444 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368\": container with ID starting with 53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368 not found: ID does not exist" containerID="53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368" Jun 02 19:59:13.122509 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.122465 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368"} err="failed to get container status \"53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368\": rpc error: code = NotFound desc = could not find container \"53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368\": container with ID starting with 53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368 not found: ID does not exist" Jun 02 19:59:13.122509 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.122480 2548 scope.go:117] "RemoveContainer" containerID="2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e" Jun 02 19:59:13.122854 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:59:13.122719 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e\": container with ID starting with 2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e not found: ID does not exist" containerID="2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e" Jun 02 19:59:13.122854 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.122845 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e"} err="failed to get container status \"2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e\": rpc error: code = NotFound desc = could not find container \"2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e\": container with ID starting with 2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e not found: ID does not exist" Jun 02 19:59:13.123051 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.122877 2548 scope.go:117] "RemoveContainer" containerID="6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d" Jun 02 19:59:13.123166 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:59:13.123149 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d\": container with ID starting with 6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d not found: ID does not exist" containerID="6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d" Jun 02 19:59:13.123216 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.123171 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d"} err="failed to get container status \"6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d\": rpc error: code = NotFound desc = could not find container \"6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d\": container with ID starting with 6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d not found: ID does not exist" Jun 02 19:59:13.123216 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.123186 2548 scope.go:117] "RemoveContainer" containerID="5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e" Jun 02 19:59:13.123402 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:59:13.123383 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e\": container with ID starting with 5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e not found: ID does not exist" containerID="5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e" Jun 02 19:59:13.123480 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.123410 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e"} err="failed to get container status \"5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e\": rpc error: code = NotFound desc = could not find container \"5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e\": container with ID starting with 5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e not found: ID does not exist" Jun 02 19:59:13.123480 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.123432 2548 scope.go:117] "RemoveContainer" containerID="53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd" Jun 02 19:59:13.123683 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:59:13.123667 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd\": container with ID starting with 53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd not found: ID does not exist" containerID="53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd" Jun 02 19:59:13.123733 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.123687 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd"} err="failed to get container status \"53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd\": rpc error: code = NotFound desc = could not find container \"53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd\": container with ID starting with 53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd not found: ID does not exist" Jun 02 19:59:13.123733 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.123701 2548 scope.go:117] "RemoveContainer" containerID="2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f" Jun 02 19:59:13.123943 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:59:13.123926 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f\": container with ID starting with 2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f not found: ID does not exist" containerID="2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f" Jun 02 19:59:13.123995 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.123953 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f"} err="failed to get container status \"2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f\": rpc error: code = NotFound desc = could not find container \"2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f\": container with ID starting with 2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f not found: ID does not exist" Jun 02 19:59:13.123995 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.123965 2548 scope.go:117] "RemoveContainer" containerID="6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18" Jun 02 19:59:13.124292 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.124178 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18"} err="failed to get container status \"6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18\": rpc error: code = NotFound desc = could not find container \"6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18\": container with ID starting with 6f3389f812ff4b5051b0699549bc739aa35bf6c22fea311a152744b9259d0c18 not found: ID does not exist" Jun 02 19:59:13.124292 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.124194 2548 scope.go:117] "RemoveContainer" containerID="53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368" Jun 02 19:59:13.124478 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.124457 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368"} err="failed to get container status \"53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368\": rpc error: code = NotFound desc = could not find container \"53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368\": container with ID starting with 53039bf79564b8c721945f30a45a482a8d33843c7a5159d8b27e41b83b7cd368 not found: ID does not exist" Jun 02 19:59:13.124538 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.124478 2548 scope.go:117] "RemoveContainer" containerID="2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e" Jun 02 19:59:13.124694 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.124677 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e"} err="failed to get container status \"2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e\": rpc error: code = NotFound desc = could not find container \"2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e\": container with ID starting with 2238a0790c5393240365df89d29076c4d9df746d47a51dee04f5b7ad73b05b2e not found: ID does not exist" Jun 02 19:59:13.124764 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.124695 2548 scope.go:117] "RemoveContainer" containerID="6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d" Jun 02 19:59:13.124963 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.124948 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d"} err="failed to get container status \"6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d\": rpc error: code = NotFound desc = could not find container \"6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d\": container with ID starting with 6f2948c01bfaa966e3cfc1b9f172dc7b063f29d37030676f5f4537750da8e21d not found: ID does not exist" Jun 02 19:59:13.124963 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.124962 2548 scope.go:117] "RemoveContainer" containerID="5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e" Jun 02 19:59:13.125165 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.125144 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e"} err="failed to get container status \"5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e\": rpc error: code = NotFound desc = could not find container \"5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e\": container with ID starting with 5b3997c3ef702bb69ce8cd52fba800b3d6672247f2c4748641d7bfabf66ba03e not found: ID does not exist" Jun 02 19:59:13.125212 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.125165 2548 scope.go:117] "RemoveContainer" containerID="53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd" Jun 02 19:59:13.125362 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.125342 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd"} err="failed to get container status \"53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd\": rpc error: code = NotFound desc = could not find container \"53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd\": container with ID starting with 53f379b6c7f1743245cbacfb44e5656c3e47f75f499cea8a1f1d9477d3dcd9fd not found: ID does not exist" Jun 02 19:59:13.125439 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.125363 2548 scope.go:117] "RemoveContainer" containerID="2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f" Jun 02 19:59:13.125575 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.125557 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f"} err="failed to get container status \"2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f\": rpc error: code = NotFound desc = could not find container \"2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f\": container with ID starting with 2b9e3cbd53deba8b76905ce5937b9b5ff425d997fd7cc407e60dfe6461f6148f not found: ID does not exist" Jun 02 19:59:13.185155 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185067 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:59:13.185400 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185386 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="alertmanager" Jun 02 19:59:13.185400 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185399 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="alertmanager" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185409 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="init-config-reloader" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185416 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="init-config-reloader" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185421 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="kube-rbac-proxy" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185427 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="kube-rbac-proxy" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185437 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58c985a-62d6-4fa6-85f2-a79d3cebac80" containerName="registry" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185442 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58c985a-62d6-4fa6-85f2-a79d3cebac80" containerName="registry" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185449 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="kube-rbac-proxy-metric" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185454 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="kube-rbac-proxy-metric" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185461 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="kube-rbac-proxy-web" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185466 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="kube-rbac-proxy-web" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185474 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="config-reloader" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185479 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="config-reloader" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185488 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="prom-label-proxy" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185493 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="prom-label-proxy" Jun 02 19:59:13.185537 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185541 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="d58c985a-62d6-4fa6-85f2-a79d3cebac80" containerName="registry" Jun 02 19:59:13.186302 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185550 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="alertmanager" Jun 02 19:59:13.186302 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185558 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="config-reloader" Jun 02 19:59:13.186302 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185564 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="kube-rbac-proxy-metric" Jun 02 19:59:13.186302 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185571 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="kube-rbac-proxy-web" Jun 02 19:59:13.186302 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185577 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="prom-label-proxy" Jun 02 19:59:13.186302 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.185584 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" containerName="kube-rbac-proxy" Jun 02 19:59:13.191146 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.191122 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.194159 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.194134 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 02 19:59:13.194722 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.194700 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 02 19:59:13.194841 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.194824 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 02 19:59:13.194893 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.194879 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 02 19:59:13.195556 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.195537 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 02 19:59:13.195556 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.195554 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 02 19:59:13.195704 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.195558 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 02 19:59:13.196700 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.196683 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 02 19:59:13.197941 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.197799 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vg66d\"" Jun 02 19:59:13.202956 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.202930 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 02 19:59:13.217589 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.217552 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:59:13.317238 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.317200 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65c75fd0-360c-4d5d-8276-c8dff3956b60" path="/var/lib/kubelet/pods/65c75fd0-360c-4d5d-8276-c8dff3956b60/volumes" Jun 02 19:59:13.332685 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.332633 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.332685 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.332689 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-config-volume\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.332922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.332709 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.332922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.332807 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bfe250e1-be31-41b6-8071-0b2d383f198b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.332922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.332839 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.332922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.332869 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bfe250e1-be31-41b6-8071-0b2d383f198b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.332922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.332897 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-web-config\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.332922 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.332911 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59m4h\" (UniqueName: \"kubernetes.io/projected/bfe250e1-be31-41b6-8071-0b2d383f198b-kube-api-access-59m4h\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.333098 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.332944 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bfe250e1-be31-41b6-8071-0b2d383f198b-config-out\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.333098 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.332978 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.333098 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.333011 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.333098 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.333038 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfe250e1-be31-41b6-8071-0b2d383f198b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.333098 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.333055 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfe250e1-be31-41b6-8071-0b2d383f198b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.434349 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.434305 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.434349 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.434357 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-config-volume\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.434597 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.434374 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.434597 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.434395 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bfe250e1-be31-41b6-8071-0b2d383f198b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.434597 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.434414 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.434597 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.434546 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bfe250e1-be31-41b6-8071-0b2d383f198b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.434822 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.434602 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-web-config\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.434822 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.434632 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-59m4h\" (UniqueName: \"kubernetes.io/projected/bfe250e1-be31-41b6-8071-0b2d383f198b-kube-api-access-59m4h\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.434822 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.434678 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bfe250e1-be31-41b6-8071-0b2d383f198b-config-out\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.434822 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.434704 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.435012 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.434921 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bfe250e1-be31-41b6-8071-0b2d383f198b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.435419 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.435115 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.435419 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.435169 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfe250e1-be31-41b6-8071-0b2d383f198b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.435419 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.435198 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfe250e1-be31-41b6-8071-0b2d383f198b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.435968 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.435945 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfe250e1-be31-41b6-8071-0b2d383f198b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.437544 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.437514 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bfe250e1-be31-41b6-8071-0b2d383f198b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.437650 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.437553 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.437719 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.437652 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-config-volume\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.437719 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.437672 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-web-config\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.437859 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.437816 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.437859 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.437828 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.438375 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.438354 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bfe250e1-be31-41b6-8071-0b2d383f198b-config-out\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.438375 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.438367 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfe250e1-be31-41b6-8071-0b2d383f198b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.438479 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.438406 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.439425 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.439407 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bfe250e1-be31-41b6-8071-0b2d383f198b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.450351 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.450323 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59m4h\" (UniqueName: \"kubernetes.io/projected/bfe250e1-be31-41b6-8071-0b2d383f198b-kube-api-access-59m4h\") pod \"alertmanager-main-0\" (UID: \"bfe250e1-be31-41b6-8071-0b2d383f198b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.500919 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.500885 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:59:13.646395 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:13.646360 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:59:13.748520 ip-10-0-129-157 kubenswrapper[2548]: E0602 19:59:13.748489 2548 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe250e1_be31_41b6_8071_0b2d383f198b.slice/crio-5d594d7b67d2ef148eb31bbdbd246d65b9941ca55ec65ae4be8c32a4f638e442.scope\": RecentStats: unable to find data in memory cache]" Jun 02 19:59:14.072710 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:14.072607 2548 generic.go:358] "Generic (PLEG): container finished" podID="bfe250e1-be31-41b6-8071-0b2d383f198b" containerID="5d594d7b67d2ef148eb31bbdbd246d65b9941ca55ec65ae4be8c32a4f638e442" exitCode=0 Jun 02 19:59:14.072710 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:14.072665 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfe250e1-be31-41b6-8071-0b2d383f198b","Type":"ContainerDied","Data":"5d594d7b67d2ef148eb31bbdbd246d65b9941ca55ec65ae4be8c32a4f638e442"} Jun 02 19:59:14.072710 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:14.072690 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfe250e1-be31-41b6-8071-0b2d383f198b","Type":"ContainerStarted","Data":"96dd6698c3cc69cbb80d223b52b9d419f39bf57a5ad80b5946fb7d44bb905e53"} Jun 02 19:59:15.079281 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.079245 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfe250e1-be31-41b6-8071-0b2d383f198b","Type":"ContainerStarted","Data":"d5faa1c5b93c6ae517aa50cdf7f0b6bfc8092d8934e5a456ab62ee1876c4f559"} Jun 02 19:59:15.079281 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.079279 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfe250e1-be31-41b6-8071-0b2d383f198b","Type":"ContainerStarted","Data":"9fb1b559eac9baec46405e62255c89df5da951f34de1aae5babbcf5fb54bb75c"} Jun 02 19:59:15.079281 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.079289 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfe250e1-be31-41b6-8071-0b2d383f198b","Type":"ContainerStarted","Data":"8b953edfbaf130fded6eebd716df9f8c9cefaee8f09f0697fe77be87057be0c1"} Jun 02 19:59:15.079895 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.079298 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfe250e1-be31-41b6-8071-0b2d383f198b","Type":"ContainerStarted","Data":"e0ca2bfcec67022755eea60170d49de8d2cfe9a18427dd9e0398e0551f8b07e8"} Jun 02 19:59:15.079895 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.079307 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfe250e1-be31-41b6-8071-0b2d383f198b","Type":"ContainerStarted","Data":"7274871d0835af69d0af886d2d7d32e2c768ab9b4341b9fe688e6499dd9446b5"} Jun 02 19:59:15.079895 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.079315 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfe250e1-be31-41b6-8071-0b2d383f198b","Type":"ContainerStarted","Data":"0d4d107204c7a518f90eecc781b71ece3a21c5bd925e1f63c0468925d929c9bc"} Jun 02 19:59:15.117191 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.117132 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.117087965 podStartE2EDuration="2.117087965s" podCreationTimestamp="2026-06-02 19:59:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:59:15.114362853 +0000 UTC m=+242.431576107" watchObservedRunningTime="2026-06-02 19:59:15.117087965 +0000 UTC m=+242.434301218" Jun 02 19:59:15.457435 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.457391 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-74857bcbf9-lhk8x"] Jun 02 19:59:15.461252 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.461228 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.464261 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.464236 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 02 19:59:15.466617 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.466583 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wpml4\"" Jun 02 19:59:15.468387 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.468364 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 02 19:59:15.468530 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.468388 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 02 19:59:15.468530 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.468480 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 02 19:59:15.469110 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.468765 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 02 19:59:15.471129 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.471108 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 02 19:59:15.483991 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.483949 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74857bcbf9-lhk8x"] Jun 02 19:59:15.554688 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.554652 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m75bt\" (UniqueName: \"kubernetes.io/projected/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-kube-api-access-m75bt\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.554918 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.554702 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-federate-client-tls\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.554918 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.554729 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.554918 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.554785 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-metrics-client-ca\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.554918 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.554895 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-telemeter-client-tls\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.555120 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.554921 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-secret-telemeter-client\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.555120 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.554949 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.555120 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.554986 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-serving-certs-ca-bundle\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.655808 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.655768 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-secret-telemeter-client\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.656017 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.655822 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.656017 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.655856 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-serving-certs-ca-bundle\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.656125 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.656058 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m75bt\" (UniqueName: \"kubernetes.io/projected/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-kube-api-access-m75bt\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.656125 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.656098 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-federate-client-tls\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.656238 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.656120 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.656238 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.656151 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-metrics-client-ca\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.656470 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.656437 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-telemeter-client-tls\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.656719 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.656694 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-serving-certs-ca-bundle\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.656891 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.656870 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-metrics-client-ca\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.656999 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.656958 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.658979 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.658953 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.659083 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.659053 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-telemeter-client-tls\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.659083 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.659069 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-secret-telemeter-client\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.659414 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.659394 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-federate-client-tls\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.669209 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.669177 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m75bt\" (UniqueName: \"kubernetes.io/projected/18f8431c-27c3-4cb4-ab0f-b6c5b7679138-kube-api-access-m75bt\") pod \"telemeter-client-74857bcbf9-lhk8x\" (UID: \"18f8431c-27c3-4cb4-ab0f-b6c5b7679138\") " pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.772865 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.772774 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" Jun 02 19:59:15.914618 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.914584 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:59:15.915226 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.915076 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="prometheus" containerID="cri-o://bb126eea9bf415595c74c96e4af3b713ba37321b79ddc95cef1bee478f960999" gracePeriod=600 Jun 02 19:59:15.915226 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.915100 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="kube-rbac-proxy" containerID="cri-o://d86dc665b6ae7441ac99331764fc128548de6ce893b9f37208f11afdf8a8f5db" gracePeriod=600 Jun 02 19:59:15.915226 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.915136 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="config-reloader" containerID="cri-o://45d11b74a95bcefcdb091cfedd78a8ee668443fe396d0aa90bf89966309e368b" gracePeriod=600 Jun 02 19:59:15.915226 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.915141 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="thanos-sidecar" containerID="cri-o://3f1e1cee45bbfef5b4bfc7d74a1b67a53b50b0fe894b0631a2dfd63de1d00981" gracePeriod=600 Jun 02 19:59:15.915226 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.915164 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2aa396d95d0d7a2ad0f35accc25be2796c90ce25da34179e8b1dd82c2d5f77d3" gracePeriod=600 Jun 02 19:59:15.915226 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.915168 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="kube-rbac-proxy-web" containerID="cri-o://564407e3a5bf3cfd68cf4886b2c38138be883ee38244bdfb547ebf63029b0f43" gracePeriod=600 Jun 02 19:59:15.931052 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:15.930716 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74857bcbf9-lhk8x"] Jun 02 19:59:15.934325 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:59:15.934285 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f8431c_27c3_4cb4_ab0f_b6c5b7679138.slice/crio-1cbf86765cf0834a1f38e96ef7fbeb2420af00560152308cefdb46e1c01db266 WatchSource:0}: Error finding container 1cbf86765cf0834a1f38e96ef7fbeb2420af00560152308cefdb46e1c01db266: Status 404 returned error can't find the container with id 1cbf86765cf0834a1f38e96ef7fbeb2420af00560152308cefdb46e1c01db266 Jun 02 19:59:16.083911 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.083865 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" event={"ID":"18f8431c-27c3-4cb4-ab0f-b6c5b7679138","Type":"ContainerStarted","Data":"1cbf86765cf0834a1f38e96ef7fbeb2420af00560152308cefdb46e1c01db266"} Jun 02 19:59:16.087036 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.087009 2548 generic.go:358] "Generic (PLEG): container finished" podID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerID="2aa396d95d0d7a2ad0f35accc25be2796c90ce25da34179e8b1dd82c2d5f77d3" exitCode=0 Jun 02 19:59:16.087036 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.087031 2548 generic.go:358] "Generic (PLEG): container finished" podID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerID="d86dc665b6ae7441ac99331764fc128548de6ce893b9f37208f11afdf8a8f5db" exitCode=0 Jun 02 19:59:16.087036 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.087037 2548 generic.go:358] "Generic (PLEG): container finished" podID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerID="564407e3a5bf3cfd68cf4886b2c38138be883ee38244bdfb547ebf63029b0f43" exitCode=0 Jun 02 19:59:16.087036 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.087043 2548 generic.go:358] "Generic (PLEG): container finished" podID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerID="3f1e1cee45bbfef5b4bfc7d74a1b67a53b50b0fe894b0631a2dfd63de1d00981" exitCode=0 Jun 02 19:59:16.087273 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.087048 2548 generic.go:358] "Generic (PLEG): container finished" podID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerID="45d11b74a95bcefcdb091cfedd78a8ee668443fe396d0aa90bf89966309e368b" exitCode=0 Jun 02 19:59:16.087273 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.087054 2548 generic.go:358] "Generic (PLEG): container finished" podID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerID="bb126eea9bf415595c74c96e4af3b713ba37321b79ddc95cef1bee478f960999" exitCode=0 Jun 02 19:59:16.087273 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.087074 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerDied","Data":"2aa396d95d0d7a2ad0f35accc25be2796c90ce25da34179e8b1dd82c2d5f77d3"} Jun 02 19:59:16.087273 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.087105 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerDied","Data":"d86dc665b6ae7441ac99331764fc128548de6ce893b9f37208f11afdf8a8f5db"} Jun 02 19:59:16.087273 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.087118 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerDied","Data":"564407e3a5bf3cfd68cf4886b2c38138be883ee38244bdfb547ebf63029b0f43"} Jun 02 19:59:16.087273 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.087132 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerDied","Data":"3f1e1cee45bbfef5b4bfc7d74a1b67a53b50b0fe894b0631a2dfd63de1d00981"} Jun 02 19:59:16.087273 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.087144 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerDied","Data":"45d11b74a95bcefcdb091cfedd78a8ee668443fe396d0aa90bf89966309e368b"} Jun 02 19:59:16.087273 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.087158 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerDied","Data":"bb126eea9bf415595c74c96e4af3b713ba37321b79ddc95cef1bee478f960999"} Jun 02 19:59:16.170952 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.170928 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:16.262911 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.262876 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-thanos-prometheus-http-client-file\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.262911 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.262918 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-kube-rbac-proxy\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263148 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.262945 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-k8s-rulefiles-0\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263148 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.262967 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-metrics-client-ca\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263148 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.262995 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-grpc-tls\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263148 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263020 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-config\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263148 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263048 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-trusted-ca-bundle\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263148 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263091 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263148 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263116 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-web-config\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263148 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263144 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-tls-assets\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263535 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263181 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263535 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263208 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tt6rq\" (UniqueName: \"kubernetes.io/projected/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-kube-api-access-tt6rq\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263535 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263246 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-serving-certs-ca-bundle\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263535 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263275 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-tls\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263535 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263325 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-kubelet-serving-ca-bundle\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263535 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263372 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-config-out\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263535 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263426 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-metrics-client-certs\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.263535 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263434 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:59:16.263535 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263462 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:59:16.263535 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263470 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-k8s-db\") pod \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\" (UID: \"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0\") " Jun 02 19:59:16.264029 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.263725 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:59:16.264029 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.264015 2548 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-metrics-client-ca\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.264136 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.264036 2548 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.264136 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.264052 2548 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.264290 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.264146 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:59:16.267671 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.265609 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-config" (OuterVolumeSpecName: "config") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:16.267993 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.267972 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-kube-api-access-tt6rq" (OuterVolumeSpecName: "kube-api-access-tt6rq") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "kube-api-access-tt6rq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:59:16.268606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.268579 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:59:16.269221 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.268958 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:16.269221 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.268994 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-config-out" (OuterVolumeSpecName: "config-out") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:59:16.269518 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.269483 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:16.269732 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.269690 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:59:16.269732 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.269688 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:16.269920 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.269780 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:16.271339 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.271311 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:16.271647 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.271624 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:59:16.271730 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.271706 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:16.271818 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.271794 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:16.285205 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.285127 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-web-config" (OuterVolumeSpecName: "web-config") pod "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" (UID: "44082f80-c134-4f39-a9ab-1a9f3f7dc9d0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:59:16.365379 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365342 2548 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-grpc-tls\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365379 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365373 2548 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365379 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365383 2548 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365392 2548 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-web-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365401 2548 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-tls-assets\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365409 2548 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365419 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tt6rq\" (UniqueName: \"kubernetes.io/projected/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-kube-api-access-tt6rq\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365428 2548 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365438 2548 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365449 2548 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-config-out\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365458 2548 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-metrics-client-certs\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365467 2548 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-k8s-db\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365475 2548 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365484 2548 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-secret-kube-rbac-proxy\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:16.365606 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:16.365492 2548 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 19:59:17.095679 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.095578 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44082f80-c134-4f39-a9ab-1a9f3f7dc9d0","Type":"ContainerDied","Data":"01159e8e8a9b292f78aac7db815f12629ab836edc522c7590fdfe680492e04d7"} Jun 02 19:59:17.095679 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.095639 2548 scope.go:117] "RemoveContainer" containerID="2aa396d95d0d7a2ad0f35accc25be2796c90ce25da34179e8b1dd82c2d5f77d3" Jun 02 19:59:17.096154 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.095721 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.105325 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.105193 2548 scope.go:117] "RemoveContainer" containerID="d86dc665b6ae7441ac99331764fc128548de6ce893b9f37208f11afdf8a8f5db" Jun 02 19:59:17.113582 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.113538 2548 scope.go:117] "RemoveContainer" containerID="564407e3a5bf3cfd68cf4886b2c38138be883ee38244bdfb547ebf63029b0f43" Jun 02 19:59:17.122059 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.122034 2548 scope.go:117] "RemoveContainer" containerID="3f1e1cee45bbfef5b4bfc7d74a1b67a53b50b0fe894b0631a2dfd63de1d00981" Jun 02 19:59:17.128160 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.128130 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:59:17.132078 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.132052 2548 scope.go:117] "RemoveContainer" containerID="45d11b74a95bcefcdb091cfedd78a8ee668443fe396d0aa90bf89966309e368b" Jun 02 19:59:17.133890 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.133863 2548 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:59:17.141329 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.141307 2548 scope.go:117] "RemoveContainer" containerID="bb126eea9bf415595c74c96e4af3b713ba37321b79ddc95cef1bee478f960999" Jun 02 19:59:17.150430 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.150413 2548 scope.go:117] "RemoveContainer" containerID="a9c73e4f5ce324961f4bd132b8b5a1dfcce5cd1755b11249c988149229d2d441" Jun 02 19:59:17.184459 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184424 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:59:17.184832 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184818 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="kube-rbac-proxy-web" Jun 02 19:59:17.184832 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184833 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="kube-rbac-proxy-web" Jun 02 19:59:17.184907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184843 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="prometheus" Jun 02 19:59:17.184907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184849 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="prometheus" Jun 02 19:59:17.184907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184858 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="thanos-sidecar" Jun 02 19:59:17.184907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184863 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="thanos-sidecar" Jun 02 19:59:17.184907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184875 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="init-config-reloader" Jun 02 19:59:17.184907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184880 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="init-config-reloader" Jun 02 19:59:17.184907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184886 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="kube-rbac-proxy-thanos" Jun 02 19:59:17.184907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184890 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="kube-rbac-proxy-thanos" Jun 02 19:59:17.184907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184898 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="config-reloader" Jun 02 19:59:17.184907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184902 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="config-reloader" Jun 02 19:59:17.184907 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184909 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="kube-rbac-proxy" Jun 02 19:59:17.185285 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184915 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="kube-rbac-proxy" Jun 02 19:59:17.185285 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184969 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="kube-rbac-proxy" Jun 02 19:59:17.185285 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184979 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="thanos-sidecar" Jun 02 19:59:17.185285 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184986 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="kube-rbac-proxy-web" Jun 02 19:59:17.185285 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.184992 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="kube-rbac-proxy-thanos" Jun 02 19:59:17.185285 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.185000 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="prometheus" Jun 02 19:59:17.185285 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.185006 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" containerName="config-reloader" Jun 02 19:59:17.190687 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.190665 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.195517 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.195491 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 02 19:59:17.195664 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.195491 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 02 19:59:17.195848 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.195829 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 02 19:59:17.196034 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.195994 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 02 19:59:17.196183 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.196160 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 02 19:59:17.196264 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.196171 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 02 19:59:17.196264 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.196172 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 02 19:59:17.196489 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.196448 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fvknc94923rfs\"" Jun 02 19:59:17.196600 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.196584 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 02 19:59:17.196838 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.196807 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pvcrv\"" Jun 02 19:59:17.196959 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.196903 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 02 19:59:17.199920 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.199667 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 02 19:59:17.201376 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.201356 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 02 19:59:17.209436 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.209402 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 02 19:59:17.212788 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.212734 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:59:17.273620 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.273584 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb8pr\" (UniqueName: \"kubernetes.io/projected/1aa43051-e1c9-4551-901d-2ae0d32143b5-kube-api-access-zb8pr\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.273799 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.273625 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.273799 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.273731 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.273900 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.273834 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.273900 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.273880 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274041 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274013 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274095 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274055 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274095 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274079 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1aa43051-e1c9-4551-901d-2ae0d32143b5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274200 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274100 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274200 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274147 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-config\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274200 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274190 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-web-config\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274335 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274217 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274335 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274251 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274335 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274287 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274335 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274323 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274518 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274355 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1aa43051-e1c9-4551-901d-2ae0d32143b5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274518 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274393 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1aa43051-e1c9-4551-901d-2ae0d32143b5-config-out\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.274518 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.274431 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.318420 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.318378 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44082f80-c134-4f39-a9ab-1a9f3f7dc9d0" path="/var/lib/kubelet/pods/44082f80-c134-4f39-a9ab-1a9f3f7dc9d0/volumes" Jun 02 19:59:17.375875 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.375779 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-web-config\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.375875 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.375825 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.375875 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.375856 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376140 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.375893 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376140 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.375918 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376140 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.375950 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1aa43051-e1c9-4551-901d-2ae0d32143b5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376140 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.375989 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1aa43051-e1c9-4551-901d-2ae0d32143b5-config-out\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376140 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376025 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376140 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376080 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zb8pr\" (UniqueName: \"kubernetes.io/projected/1aa43051-e1c9-4551-901d-2ae0d32143b5-kube-api-access-zb8pr\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376140 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376126 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376463 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376185 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376463 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376220 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376463 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376264 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376463 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376338 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376463 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376364 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376463 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376393 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1aa43051-e1c9-4551-901d-2ae0d32143b5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376463 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376419 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376463 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376454 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-config\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.376886 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376821 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.377017 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.376986 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.377236 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.377210 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.377904 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.377874 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.379869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.379132 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.379869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.379276 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-config\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.379869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.379418 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1aa43051-e1c9-4551-901d-2ae0d32143b5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.379869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.379425 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-web-config\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.379869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.379604 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.379869 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.379625 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1aa43051-e1c9-4551-901d-2ae0d32143b5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.381037 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.380136 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1aa43051-e1c9-4551-901d-2ae0d32143b5-config-out\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.381037 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.380262 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1aa43051-e1c9-4551-901d-2ae0d32143b5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.381037 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.380573 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.381792 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.381768 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.381896 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.381807 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.382118 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.382098 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.382184 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.382116 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1aa43051-e1c9-4551-901d-2ae0d32143b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.389394 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.389372 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb8pr\" (UniqueName: \"kubernetes.io/projected/1aa43051-e1c9-4551-901d-2ae0d32143b5-kube-api-access-zb8pr\") pod \"prometheus-k8s-0\" (UID: \"1aa43051-e1c9-4551-901d-2ae0d32143b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:17.502824 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:17.502782 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:18.066749 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:18.066711 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:59:18.069700 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:59:18.069672 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa43051_e1c9_4551_901d_2ae0d32143b5.slice/crio-fa4b6c72b387ae8a534ab7f011eb020abe1d4b05e2e3184d69ab35874af767aa WatchSource:0}: Error finding container fa4b6c72b387ae8a534ab7f011eb020abe1d4b05e2e3184d69ab35874af767aa: Status 404 returned error can't find the container with id fa4b6c72b387ae8a534ab7f011eb020abe1d4b05e2e3184d69ab35874af767aa Jun 02 19:59:18.100368 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:18.100337 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" event={"ID":"18f8431c-27c3-4cb4-ab0f-b6c5b7679138","Type":"ContainerStarted","Data":"e4c3f1e1991e4d082296ef2cbcce08e68572d117024775830c6dc15c30a00f99"} Jun 02 19:59:18.101334 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:18.101307 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1aa43051-e1c9-4551-901d-2ae0d32143b5","Type":"ContainerStarted","Data":"fa4b6c72b387ae8a534ab7f011eb020abe1d4b05e2e3184d69ab35874af767aa"} Jun 02 19:59:19.109659 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:19.109616 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" event={"ID":"18f8431c-27c3-4cb4-ab0f-b6c5b7679138","Type":"ContainerStarted","Data":"c03a9f91c9b8329620d823327c5a773b454a9a68a25a87d7c4e28e4efc567335"} Jun 02 19:59:19.109659 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:19.109659 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" event={"ID":"18f8431c-27c3-4cb4-ab0f-b6c5b7679138","Type":"ContainerStarted","Data":"d055b74c38104513c1419ac40873caa43907f64e95f68aa790af36085af8af26"} Jun 02 19:59:19.110935 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:19.110911 2548 generic.go:358] "Generic (PLEG): container finished" podID="1aa43051-e1c9-4551-901d-2ae0d32143b5" containerID="2ae3f05464ccf7cdc3bbca8eefa8b7b6d1c3d161b54609e585beb5baf67e7f83" exitCode=0 Jun 02 19:59:19.111055 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:19.110997 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1aa43051-e1c9-4551-901d-2ae0d32143b5","Type":"ContainerDied","Data":"2ae3f05464ccf7cdc3bbca8eefa8b7b6d1c3d161b54609e585beb5baf67e7f83"} Jun 02 19:59:19.135816 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:19.135760 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-74857bcbf9-lhk8x" podStartSLOduration=2.180111357 podStartE2EDuration="4.135721992s" podCreationTimestamp="2026-06-02 19:59:15 +0000 UTC" firstStartedPulling="2026-06-02 19:59:16.02149434 +0000 UTC m=+243.338707572" lastFinishedPulling="2026-06-02 19:59:17.977104958 +0000 UTC m=+245.294318207" observedRunningTime="2026-06-02 19:59:19.135252034 +0000 UTC m=+246.452465312" watchObservedRunningTime="2026-06-02 19:59:19.135721992 +0000 UTC m=+246.452935247" Jun 02 19:59:20.116939 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:20.116897 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1aa43051-e1c9-4551-901d-2ae0d32143b5","Type":"ContainerStarted","Data":"8a39e9eaa44ea4f28ddd73ce1dfe9de28c4efa9818e517368f7f8805a770a8a6"} Jun 02 19:59:20.116939 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:20.116944 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1aa43051-e1c9-4551-901d-2ae0d32143b5","Type":"ContainerStarted","Data":"00ff0366f6fc2116cf25abea36d2f227d4c3f92821ef3a0bad73771f183b5b0a"} Jun 02 19:59:20.117446 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:20.116956 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1aa43051-e1c9-4551-901d-2ae0d32143b5","Type":"ContainerStarted","Data":"69db4dc709f78c05b63df797c52865767ebee0876685895308591c312af4a5bd"} Jun 02 19:59:20.117446 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:20.116968 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1aa43051-e1c9-4551-901d-2ae0d32143b5","Type":"ContainerStarted","Data":"ccb4b0b1949905eb0133436216dd0d03b9d4503e774cb40d2406bf249b477747"} Jun 02 19:59:20.117446 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:20.116981 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1aa43051-e1c9-4551-901d-2ae0d32143b5","Type":"ContainerStarted","Data":"0ad70d0b1631458bff77b27c3e9907dee55efcdde67725dff2f909a2009b22f0"} Jun 02 19:59:20.117446 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:20.116992 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1aa43051-e1c9-4551-901d-2ae0d32143b5","Type":"ContainerStarted","Data":"7a343fc1d0f7893ca642f9f535ec6b2e2af9f57426d4e05391c12dbda02fbd6e"} Jun 02 19:59:20.165003 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:20.164947 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.164929833 podStartE2EDuration="3.164929833s" podCreationTimestamp="2026-06-02 19:59:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:59:20.164420228 +0000 UTC m=+247.481633483" watchObservedRunningTime="2026-06-02 19:59:20.164929833 +0000 UTC m=+247.482143088" Jun 02 19:59:22.503050 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:22.503002 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:59:25.150655 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:25.150616 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:59:25.152988 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:25.152966 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c848a2cb-1edf-4aa2-b593-c0f19d847476-metrics-certs\") pod \"network-metrics-daemon-wwpfn\" (UID: \"c848a2cb-1edf-4aa2-b593-c0f19d847476\") " pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:59:25.416910 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:25.416826 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r9fgf\"" Jun 02 19:59:25.425082 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:25.425058 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wwpfn" Jun 02 19:59:25.558920 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:25.558886 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wwpfn"] Jun 02 19:59:25.562094 ip-10-0-129-157 kubenswrapper[2548]: W0602 19:59:25.562066 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc848a2cb_1edf_4aa2_b593_c0f19d847476.slice/crio-4f91909a827d8c45390c4ca05e2b292ead4dfbf50dd44a88f3f2e91733856206 WatchSource:0}: Error finding container 4f91909a827d8c45390c4ca05e2b292ead4dfbf50dd44a88f3f2e91733856206: Status 404 returned error can't find the container with id 4f91909a827d8c45390c4ca05e2b292ead4dfbf50dd44a88f3f2e91733856206 Jun 02 19:59:26.139119 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:26.139080 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wwpfn" event={"ID":"c848a2cb-1edf-4aa2-b593-c0f19d847476","Type":"ContainerStarted","Data":"4f91909a827d8c45390c4ca05e2b292ead4dfbf50dd44a88f3f2e91733856206"} Jun 02 19:59:27.145571 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:27.145529 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wwpfn" event={"ID":"c848a2cb-1edf-4aa2-b593-c0f19d847476","Type":"ContainerStarted","Data":"f8bf21c68664e3c00f512edb315dd2bc640e5a250bb2200a3313b252d8a39957"} Jun 02 19:59:28.149628 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:28.149590 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wwpfn" event={"ID":"c848a2cb-1edf-4aa2-b593-c0f19d847476","Type":"ContainerStarted","Data":"f741db75e86cea29af65e2dc639f13bce6fbab65e0afeec3edb3606802534306"} Jun 02 19:59:28.167193 ip-10-0-129-157 kubenswrapper[2548]: I0602 19:59:28.167140 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wwpfn" podStartSLOduration=253.701413711 podStartE2EDuration="4m15.167124368s" podCreationTimestamp="2026-06-02 19:55:13 +0000 UTC" firstStartedPulling="2026-06-02 19:59:25.563844164 +0000 UTC m=+252.881057396" lastFinishedPulling="2026-06-02 19:59:27.029554806 +0000 UTC m=+254.346768053" observedRunningTime="2026-06-02 19:59:28.165372392 +0000 UTC m=+255.482585645" watchObservedRunningTime="2026-06-02 19:59:28.167124368 +0000 UTC m=+255.484337622" Jun 02 20:00:13.135076 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:00:13.135046 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:00:13.135507 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:00:13.135298 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:00:13.141398 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:00:13.141376 2548 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 20:00:17.503348 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:00:17.503283 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:00:17.519615 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:00:17.519579 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:00:18.314641 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:00:18.314611 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:02:33.403478 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.403388 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-mnhkd"] Jun 02 20:02:33.406714 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.406688 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-mnhkd" Jun 02 20:02:33.410182 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.410155 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 02 20:02:33.410309 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.410155 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-w7fmd\"" Jun 02 20:02:33.410931 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.410913 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 02 20:02:33.427371 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.427326 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-mnhkd"] Jun 02 20:02:33.499108 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.499059 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5383432e-83d2-4b6f-9455-4a659a019237-bound-sa-token\") pod \"cert-manager-79c8d999ff-mnhkd\" (UID: \"5383432e-83d2-4b6f-9455-4a659a019237\") " pod="cert-manager/cert-manager-79c8d999ff-mnhkd" Jun 02 20:02:33.499291 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.499178 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbmrn\" (UniqueName: \"kubernetes.io/projected/5383432e-83d2-4b6f-9455-4a659a019237-kube-api-access-nbmrn\") pod \"cert-manager-79c8d999ff-mnhkd\" (UID: \"5383432e-83d2-4b6f-9455-4a659a019237\") " pod="cert-manager/cert-manager-79c8d999ff-mnhkd" Jun 02 20:02:33.600212 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.600168 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5383432e-83d2-4b6f-9455-4a659a019237-bound-sa-token\") pod \"cert-manager-79c8d999ff-mnhkd\" (UID: \"5383432e-83d2-4b6f-9455-4a659a019237\") " pod="cert-manager/cert-manager-79c8d999ff-mnhkd" Jun 02 20:02:33.600403 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.600284 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nbmrn\" (UniqueName: \"kubernetes.io/projected/5383432e-83d2-4b6f-9455-4a659a019237-kube-api-access-nbmrn\") pod \"cert-manager-79c8d999ff-mnhkd\" (UID: \"5383432e-83d2-4b6f-9455-4a659a019237\") " pod="cert-manager/cert-manager-79c8d999ff-mnhkd" Jun 02 20:02:33.618450 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.618419 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5383432e-83d2-4b6f-9455-4a659a019237-bound-sa-token\") pod \"cert-manager-79c8d999ff-mnhkd\" (UID: \"5383432e-83d2-4b6f-9455-4a659a019237\") " pod="cert-manager/cert-manager-79c8d999ff-mnhkd" Jun 02 20:02:33.618716 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.618690 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbmrn\" (UniqueName: \"kubernetes.io/projected/5383432e-83d2-4b6f-9455-4a659a019237-kube-api-access-nbmrn\") pod \"cert-manager-79c8d999ff-mnhkd\" (UID: \"5383432e-83d2-4b6f-9455-4a659a019237\") " pod="cert-manager/cert-manager-79c8d999ff-mnhkd" Jun 02 20:02:33.731911 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.731824 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-mnhkd" Jun 02 20:02:33.864593 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.864566 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-mnhkd"] Jun 02 20:02:33.867460 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:02:33.867429 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5383432e_83d2_4b6f_9455_4a659a019237.slice/crio-ac91e8df742b71767ce02604909a7b703f1d8163eab7e927724ef725594a0e1f WatchSource:0}: Error finding container ac91e8df742b71767ce02604909a7b703f1d8163eab7e927724ef725594a0e1f: Status 404 returned error can't find the container with id ac91e8df742b71767ce02604909a7b703f1d8163eab7e927724ef725594a0e1f Jun 02 20:02:33.872447 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:33.872428 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 20:02:34.716542 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:34.716497 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-mnhkd" event={"ID":"5383432e-83d2-4b6f-9455-4a659a019237","Type":"ContainerStarted","Data":"ac91e8df742b71767ce02604909a7b703f1d8163eab7e927724ef725594a0e1f"} Jun 02 20:02:35.088975 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.088895 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf"] Jun 02 20:02:35.093754 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.093713 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf" Jun 02 20:02:35.096620 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.096593 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-86s9z\"" Jun 02 20:02:35.096791 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.096620 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 02 20:02:35.097586 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.097401 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 02 20:02:35.100917 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.100890 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf"] Jun 02 20:02:35.214134 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.214093 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mzhv\" (UniqueName: \"kubernetes.io/projected/1ae7b561-29d7-4743-9039-858d328baa1a-kube-api-access-9mzhv\") pod \"openshift-lws-operator-bfc7f696d-58rbf\" (UID: \"1ae7b561-29d7-4743-9039-858d328baa1a\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf" Jun 02 20:02:35.214134 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.214142 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ae7b561-29d7-4743-9039-858d328baa1a-tmp\") pod \"openshift-lws-operator-bfc7f696d-58rbf\" (UID: \"1ae7b561-29d7-4743-9039-858d328baa1a\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf" Jun 02 20:02:35.314787 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.314723 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9mzhv\" (UniqueName: \"kubernetes.io/projected/1ae7b561-29d7-4743-9039-858d328baa1a-kube-api-access-9mzhv\") pod \"openshift-lws-operator-bfc7f696d-58rbf\" (UID: \"1ae7b561-29d7-4743-9039-858d328baa1a\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf" Jun 02 20:02:35.314964 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.314800 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ae7b561-29d7-4743-9039-858d328baa1a-tmp\") pod \"openshift-lws-operator-bfc7f696d-58rbf\" (UID: \"1ae7b561-29d7-4743-9039-858d328baa1a\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf" Jun 02 20:02:35.315310 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.315269 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ae7b561-29d7-4743-9039-858d328baa1a-tmp\") pod \"openshift-lws-operator-bfc7f696d-58rbf\" (UID: \"1ae7b561-29d7-4743-9039-858d328baa1a\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf" Jun 02 20:02:35.325451 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.325416 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mzhv\" (UniqueName: \"kubernetes.io/projected/1ae7b561-29d7-4743-9039-858d328baa1a-kube-api-access-9mzhv\") pod \"openshift-lws-operator-bfc7f696d-58rbf\" (UID: \"1ae7b561-29d7-4743-9039-858d328baa1a\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf" Jun 02 20:02:35.406908 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.406871 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf" Jun 02 20:02:35.975783 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:35.975732 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf"] Jun 02 20:02:35.978858 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:02:35.978827 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ae7b561_29d7_4743_9039_858d328baa1a.slice/crio-9d18e74dbb862876fc3544ef8932f45ba0f69c77548a539e18a9eb45ea776407 WatchSource:0}: Error finding container 9d18e74dbb862876fc3544ef8932f45ba0f69c77548a539e18a9eb45ea776407: Status 404 returned error can't find the container with id 9d18e74dbb862876fc3544ef8932f45ba0f69c77548a539e18a9eb45ea776407 Jun 02 20:02:36.727539 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:36.727499 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf" event={"ID":"1ae7b561-29d7-4743-9039-858d328baa1a","Type":"ContainerStarted","Data":"9d18e74dbb862876fc3544ef8932f45ba0f69c77548a539e18a9eb45ea776407"} Jun 02 20:02:39.740363 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:39.740257 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf" event={"ID":"1ae7b561-29d7-4743-9039-858d328baa1a","Type":"ContainerStarted","Data":"42fd5cd8fb22a301c5794a64951feff194d4f8131432b548fdd9252892b3e014"} Jun 02 20:02:39.741746 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:39.741721 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-mnhkd" event={"ID":"5383432e-83d2-4b6f-9455-4a659a019237","Type":"ContainerStarted","Data":"431078e538a17d8a2bb5fec53b0a9254fd819b1c51d32cc6fa70ee20186e4c2c"} Jun 02 20:02:39.759439 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:39.759390 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-58rbf" podStartSLOduration=1.25471218 podStartE2EDuration="4.759371401s" podCreationTimestamp="2026-06-02 20:02:35 +0000 UTC" firstStartedPulling="2026-06-02 20:02:35.980344761 +0000 UTC m=+443.297557997" lastFinishedPulling="2026-06-02 20:02:39.485003983 +0000 UTC m=+446.802217218" observedRunningTime="2026-06-02 20:02:39.757955266 +0000 UTC m=+447.075168501" watchObservedRunningTime="2026-06-02 20:02:39.759371401 +0000 UTC m=+447.076584657" Jun 02 20:02:39.777851 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:39.777801 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-mnhkd" podStartSLOduration=1.217009547 podStartE2EDuration="6.777786958s" podCreationTimestamp="2026-06-02 20:02:33 +0000 UTC" firstStartedPulling="2026-06-02 20:02:33.872609179 +0000 UTC m=+441.189822411" lastFinishedPulling="2026-06-02 20:02:39.433386591 +0000 UTC m=+446.750599822" observedRunningTime="2026-06-02 20:02:39.775126885 +0000 UTC m=+447.092340140" watchObservedRunningTime="2026-06-02 20:02:39.777786958 +0000 UTC m=+447.095000212" Jun 02 20:02:55.394614 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.394574 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j"] Jun 02 20:02:55.398462 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.398443 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.402906 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.402880 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sstl7\"" Jun 02 20:02:55.403025 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.403000 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 02 20:02:55.403508 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.403484 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 02 20:02:55.403577 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.403558 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 20:02:55.403830 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.403810 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 20:02:55.419370 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.419341 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j"] Jun 02 20:02:55.499897 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.499853 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhjwf\" (UniqueName: \"kubernetes.io/projected/f43fcdad-b0ed-4457-879f-62b470c72622-kube-api-access-jhjwf\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.499897 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.499902 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f43fcdad-b0ed-4457-879f-62b470c72622-manifests\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.500133 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.499982 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f43fcdad-b0ed-4457-879f-62b470c72622-tmp\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.500133 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.500006 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f43fcdad-b0ed-4457-879f-62b470c72622-webhook-cert\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.500133 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.500072 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f43fcdad-b0ed-4457-879f-62b470c72622-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.601347 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.601302 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f43fcdad-b0ed-4457-879f-62b470c72622-manifests\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.601558 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.601371 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f43fcdad-b0ed-4457-879f-62b470c72622-tmp\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.601558 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.601397 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f43fcdad-b0ed-4457-879f-62b470c72622-webhook-cert\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.601558 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.601430 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f43fcdad-b0ed-4457-879f-62b470c72622-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.601558 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.601472 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jhjwf\" (UniqueName: \"kubernetes.io/projected/f43fcdad-b0ed-4457-879f-62b470c72622-kube-api-access-jhjwf\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.601874 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.601852 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f43fcdad-b0ed-4457-879f-62b470c72622-manifests\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.603812 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.603790 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f43fcdad-b0ed-4457-879f-62b470c72622-tmp\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.604069 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.604034 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f43fcdad-b0ed-4457-879f-62b470c72622-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.604132 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.604074 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f43fcdad-b0ed-4457-879f-62b470c72622-webhook-cert\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.613873 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.613846 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhjwf\" (UniqueName: \"kubernetes.io/projected/f43fcdad-b0ed-4457-879f-62b470c72622-kube-api-access-jhjwf\") pod \"opendatahub-operator-controller-manager-6977cb796-js76j\" (UID: \"f43fcdad-b0ed-4457-879f-62b470c72622\") " pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.709227 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.709197 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:02:55.848921 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:55.848889 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j"] Jun 02 20:02:55.851693 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:02:55.851665 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf43fcdad_b0ed_4457_879f_62b470c72622.slice/crio-c26b9a2a6f8f2998c27baa0bf370bb1994c6c2a69101bf422b51b88c46a55924 WatchSource:0}: Error finding container c26b9a2a6f8f2998c27baa0bf370bb1994c6c2a69101bf422b51b88c46a55924: Status 404 returned error can't find the container with id c26b9a2a6f8f2998c27baa0bf370bb1994c6c2a69101bf422b51b88c46a55924 Jun 02 20:02:56.808756 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:56.808705 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" event={"ID":"f43fcdad-b0ed-4457-879f-62b470c72622","Type":"ContainerStarted","Data":"c26b9a2a6f8f2998c27baa0bf370bb1994c6c2a69101bf422b51b88c46a55924"} Jun 02 20:02:59.822472 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:02:59.822433 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" event={"ID":"f43fcdad-b0ed-4457-879f-62b470c72622","Type":"ContainerStarted","Data":"ed8920475ae18a2870073df64ed6296e9de7843385a35dd910cdc34cf445e84a"} Jun 02 20:03:00.827721 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:00.827686 2548 generic.go:358] "Generic (PLEG): container finished" podID="f43fcdad-b0ed-4457-879f-62b470c72622" containerID="ed8920475ae18a2870073df64ed6296e9de7843385a35dd910cdc34cf445e84a" exitCode=0 Jun 02 20:03:00.828217 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:00.827728 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" event={"ID":"f43fcdad-b0ed-4457-879f-62b470c72622","Type":"ContainerDied","Data":"ed8920475ae18a2870073df64ed6296e9de7843385a35dd910cdc34cf445e84a"} Jun 02 20:03:01.832712 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:01.832677 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" event={"ID":"f43fcdad-b0ed-4457-879f-62b470c72622","Type":"ContainerStarted","Data":"5b07d43c26dbcfb989b14dd11202217f3f6ef24331d910b583e0dfadfd7a2fa3"} Jun 02 20:03:01.833104 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:01.832781 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:03:01.856330 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:01.856265 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" podStartSLOduration=1.564799822 podStartE2EDuration="6.856243675s" podCreationTimestamp="2026-06-02 20:02:55 +0000 UTC" firstStartedPulling="2026-06-02 20:02:55.853457311 +0000 UTC m=+463.170670543" lastFinishedPulling="2026-06-02 20:03:01.144901147 +0000 UTC m=+468.462114396" observedRunningTime="2026-06-02 20:03:01.854694651 +0000 UTC m=+469.171907917" watchObservedRunningTime="2026-06-02 20:03:01.856243675 +0000 UTC m=+469.173456931" Jun 02 20:03:12.839216 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:12.839183 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6977cb796-js76j" Jun 02 20:03:15.770021 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.769978 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-69d845485f-4m69k"] Jun 02 20:03:15.784690 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.784660 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:15.787647 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.787618 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 02 20:03:15.788580 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.788559 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 02 20:03:15.788712 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.788587 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 02 20:03:15.788712 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.788609 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bbxr4\"" Jun 02 20:03:15.790285 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.790261 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69d845485f-4m69k"] Jun 02 20:03:15.888251 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.888219 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/60887d2f-199c-4d20-b12d-e36308588b6f-manager-config\") pod \"lws-controller-manager-69d845485f-4m69k\" (UID: \"60887d2f-199c-4d20-b12d-e36308588b6f\") " pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:15.888449 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.888267 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60887d2f-199c-4d20-b12d-e36308588b6f-cert\") pod \"lws-controller-manager-69d845485f-4m69k\" (UID: \"60887d2f-199c-4d20-b12d-e36308588b6f\") " pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:15.888449 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.888338 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s79g\" (UniqueName: \"kubernetes.io/projected/60887d2f-199c-4d20-b12d-e36308588b6f-kube-api-access-4s79g\") pod \"lws-controller-manager-69d845485f-4m69k\" (UID: \"60887d2f-199c-4d20-b12d-e36308588b6f\") " pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:15.888449 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.888399 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/60887d2f-199c-4d20-b12d-e36308588b6f-metrics-cert\") pod \"lws-controller-manager-69d845485f-4m69k\" (UID: \"60887d2f-199c-4d20-b12d-e36308588b6f\") " pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:15.989183 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.989140 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/60887d2f-199c-4d20-b12d-e36308588b6f-manager-config\") pod \"lws-controller-manager-69d845485f-4m69k\" (UID: \"60887d2f-199c-4d20-b12d-e36308588b6f\") " pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:15.989386 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.989209 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60887d2f-199c-4d20-b12d-e36308588b6f-cert\") pod \"lws-controller-manager-69d845485f-4m69k\" (UID: \"60887d2f-199c-4d20-b12d-e36308588b6f\") " pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:15.989386 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.989239 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4s79g\" (UniqueName: \"kubernetes.io/projected/60887d2f-199c-4d20-b12d-e36308588b6f-kube-api-access-4s79g\") pod \"lws-controller-manager-69d845485f-4m69k\" (UID: \"60887d2f-199c-4d20-b12d-e36308588b6f\") " pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:15.989386 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.989281 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/60887d2f-199c-4d20-b12d-e36308588b6f-metrics-cert\") pod \"lws-controller-manager-69d845485f-4m69k\" (UID: \"60887d2f-199c-4d20-b12d-e36308588b6f\") " pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:15.989949 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.989921 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/60887d2f-199c-4d20-b12d-e36308588b6f-manager-config\") pod \"lws-controller-manager-69d845485f-4m69k\" (UID: \"60887d2f-199c-4d20-b12d-e36308588b6f\") " pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:15.991710 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.991687 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/60887d2f-199c-4d20-b12d-e36308588b6f-metrics-cert\") pod \"lws-controller-manager-69d845485f-4m69k\" (UID: \"60887d2f-199c-4d20-b12d-e36308588b6f\") " pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:15.991821 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:15.991753 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60887d2f-199c-4d20-b12d-e36308588b6f-cert\") pod \"lws-controller-manager-69d845485f-4m69k\" (UID: \"60887d2f-199c-4d20-b12d-e36308588b6f\") " pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:16.001267 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:16.001233 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s79g\" (UniqueName: \"kubernetes.io/projected/60887d2f-199c-4d20-b12d-e36308588b6f-kube-api-access-4s79g\") pod \"lws-controller-manager-69d845485f-4m69k\" (UID: \"60887d2f-199c-4d20-b12d-e36308588b6f\") " pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:16.095725 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:16.095624 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:16.227931 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:16.227855 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69d845485f-4m69k"] Jun 02 20:03:16.230177 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:03:16.230148 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60887d2f_199c_4d20_b12d_e36308588b6f.slice/crio-52386dd5465a361c598d2fb174c4d47c786062352082a27c5901451d0d949abf WatchSource:0}: Error finding container 52386dd5465a361c598d2fb174c4d47c786062352082a27c5901451d0d949abf: Status 404 returned error can't find the container with id 52386dd5465a361c598d2fb174c4d47c786062352082a27c5901451d0d949abf Jun 02 20:03:16.891608 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:16.891574 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" event={"ID":"60887d2f-199c-4d20-b12d-e36308588b6f","Type":"ContainerStarted","Data":"52386dd5465a361c598d2fb174c4d47c786062352082a27c5901451d0d949abf"} Jun 02 20:03:18.899884 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:18.899848 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" event={"ID":"60887d2f-199c-4d20-b12d-e36308588b6f","Type":"ContainerStarted","Data":"beba80f1b7606a829123970fae07b8e80518a4c9dcdce4150dc23008860ea543"} Jun 02 20:03:18.900287 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:18.899960 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:18.919565 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:18.919512 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" podStartSLOduration=2.301139865 podStartE2EDuration="3.919496751s" podCreationTimestamp="2026-06-02 20:03:15 +0000 UTC" firstStartedPulling="2026-06-02 20:03:16.232052334 +0000 UTC m=+483.549265565" lastFinishedPulling="2026-06-02 20:03:17.850409216 +0000 UTC m=+485.167622451" observedRunningTime="2026-06-02 20:03:18.91854422 +0000 UTC m=+486.235757478" watchObservedRunningTime="2026-06-02 20:03:18.919496751 +0000 UTC m=+486.236710004" Jun 02 20:03:29.905830 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:29.905796 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-69d845485f-4m69k" Jun 02 20:03:33.803154 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:33.803118 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-54b5c7c656-qfgpk"] Jun 02 20:03:33.806525 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:33.806503 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-54b5c7c656-qfgpk" Jun 02 20:03:33.815305 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:33.815277 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8ph4m\"" Jun 02 20:03:33.815780 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:33.815755 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 02 20:03:33.840049 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:33.840016 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-54b5c7c656-qfgpk"] Jun 02 20:03:33.864662 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:33.864627 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf7qn\" (UniqueName: \"kubernetes.io/projected/31ec5341-8c1b-437c-aa4e-130e89789768-kube-api-access-qf7qn\") pod \"dashboard-redirect-54b5c7c656-qfgpk\" (UID: \"31ec5341-8c1b-437c-aa4e-130e89789768\") " pod="opendatahub/dashboard-redirect-54b5c7c656-qfgpk" Jun 02 20:03:33.864850 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:33.864728 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/31ec5341-8c1b-437c-aa4e-130e89789768-redirect-config\") pod \"dashboard-redirect-54b5c7c656-qfgpk\" (UID: \"31ec5341-8c1b-437c-aa4e-130e89789768\") " pod="opendatahub/dashboard-redirect-54b5c7c656-qfgpk" Jun 02 20:03:33.965586 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:33.965550 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qf7qn\" (UniqueName: \"kubernetes.io/projected/31ec5341-8c1b-437c-aa4e-130e89789768-kube-api-access-qf7qn\") pod \"dashboard-redirect-54b5c7c656-qfgpk\" (UID: \"31ec5341-8c1b-437c-aa4e-130e89789768\") " pod="opendatahub/dashboard-redirect-54b5c7c656-qfgpk" Jun 02 20:03:33.965785 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:33.965628 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/31ec5341-8c1b-437c-aa4e-130e89789768-redirect-config\") pod \"dashboard-redirect-54b5c7c656-qfgpk\" (UID: \"31ec5341-8c1b-437c-aa4e-130e89789768\") " pod="opendatahub/dashboard-redirect-54b5c7c656-qfgpk" Jun 02 20:03:33.966319 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:33.966300 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/31ec5341-8c1b-437c-aa4e-130e89789768-redirect-config\") pod \"dashboard-redirect-54b5c7c656-qfgpk\" (UID: \"31ec5341-8c1b-437c-aa4e-130e89789768\") " pod="opendatahub/dashboard-redirect-54b5c7c656-qfgpk" Jun 02 20:03:33.975563 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:33.975531 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf7qn\" (UniqueName: \"kubernetes.io/projected/31ec5341-8c1b-437c-aa4e-130e89789768-kube-api-access-qf7qn\") pod \"dashboard-redirect-54b5c7c656-qfgpk\" (UID: \"31ec5341-8c1b-437c-aa4e-130e89789768\") " pod="opendatahub/dashboard-redirect-54b5c7c656-qfgpk" Jun 02 20:03:34.117852 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:34.117722 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-54b5c7c656-qfgpk" Jun 02 20:03:34.268944 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:34.268731 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-54b5c7c656-qfgpk"] Jun 02 20:03:34.955948 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:34.955913 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-54b5c7c656-qfgpk" event={"ID":"31ec5341-8c1b-437c-aa4e-130e89789768","Type":"ContainerStarted","Data":"36a648be4eafa13cf657e2fb23866e9e2fece65f09426d6931d64fd8cb730a51"} Jun 02 20:03:36.964869 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:36.964831 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-54b5c7c656-qfgpk" event={"ID":"31ec5341-8c1b-437c-aa4e-130e89789768","Type":"ContainerStarted","Data":"d3a52030bd8e13124b8db6b7bcd5607cfff64245426ea4314b00597335e97850"} Jun 02 20:03:36.983004 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:36.982945 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-54b5c7c656-qfgpk" podStartSLOduration=1.862234222 podStartE2EDuration="3.982917323s" podCreationTimestamp="2026-06-02 20:03:33 +0000 UTC" firstStartedPulling="2026-06-02 20:03:34.273707147 +0000 UTC m=+501.590920394" lastFinishedPulling="2026-06-02 20:03:36.39439025 +0000 UTC m=+503.711603495" observedRunningTime="2026-06-02 20:03:36.982098489 +0000 UTC m=+504.299311767" watchObservedRunningTime="2026-06-02 20:03:36.982917323 +0000 UTC m=+504.300130576" Jun 02 20:03:46.818554 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.818515 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf"] Jun 02 20:03:46.823276 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.823253 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.826378 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.826355 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-p76l7\"" Jun 02 20:03:46.826813 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.826788 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 20:03:46.826937 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.826815 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 02 20:03:46.826937 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.826838 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 20:03:46.834559 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.834531 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf"] Jun 02 20:03:46.886724 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.886685 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.886923 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.886771 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00a7259e-d63c-4b89-a3a7-dfc037790e60-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.886923 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.886854 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.886923 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.886907 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00a7259e-d63c-4b89-a3a7-dfc037790e60-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.887030 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.886976 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.887030 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.887020 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.887096 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.887056 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxdx9\" (UniqueName: \"kubernetes.io/projected/00a7259e-d63c-4b89-a3a7-dfc037790e60-kube-api-access-gxdx9\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.887096 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.887082 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.887161 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.887115 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00a7259e-d63c-4b89-a3a7-dfc037790e60-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.988114 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988068 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00a7259e-d63c-4b89-a3a7-dfc037790e60-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.988114 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988116 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.988348 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988248 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00a7259e-d63c-4b89-a3a7-dfc037790e60-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.988348 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988328 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.988462 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988359 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.988462 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988399 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gxdx9\" (UniqueName: \"kubernetes.io/projected/00a7259e-d63c-4b89-a3a7-dfc037790e60-kube-api-access-gxdx9\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.988462 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988436 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.988619 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988487 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00a7259e-d63c-4b89-a3a7-dfc037790e60-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.988619 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988495 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.988619 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988555 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.988888 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988864 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.989002 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988914 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.989002 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.988934 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.989098 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.989077 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00a7259e-d63c-4b89-a3a7-dfc037790e60-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.990621 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.990595 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00a7259e-d63c-4b89-a3a7-dfc037790e60-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.991011 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.990993 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00a7259e-d63c-4b89-a3a7-dfc037790e60-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.997876 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.997845 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00a7259e-d63c-4b89-a3a7-dfc037790e60-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:46.998307 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:46.998253 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxdx9\" (UniqueName: \"kubernetes.io/projected/00a7259e-d63c-4b89-a3a7-dfc037790e60-kube-api-access-gxdx9\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf\" (UID: \"00a7259e-d63c-4b89-a3a7-dfc037790e60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:47.136282 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:47.136179 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:47.282090 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:47.279591 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf"] Jun 02 20:03:47.287621 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:03:47.287589 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a7259e_d63c_4b89_a3a7_dfc037790e60.slice/crio-d711b6ba6736234dfda4264b7bac98efa6490dc3efa721fece927e3de9295eae WatchSource:0}: Error finding container d711b6ba6736234dfda4264b7bac98efa6490dc3efa721fece927e3de9295eae: Status 404 returned error can't find the container with id d711b6ba6736234dfda4264b7bac98efa6490dc3efa721fece927e3de9295eae Jun 02 20:03:48.005318 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:48.005267 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" event={"ID":"00a7259e-d63c-4b89-a3a7-dfc037790e60","Type":"ContainerStarted","Data":"d711b6ba6736234dfda4264b7bac98efa6490dc3efa721fece927e3de9295eae"} Jun 02 20:03:49.683698 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:49.683648 2548 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 02 20:03:49.684089 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:49.683757 2548 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 02 20:03:49.684089 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:49.683805 2548 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 02 20:03:50.014563 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:50.014480 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" event={"ID":"00a7259e-d63c-4b89-a3a7-dfc037790e60","Type":"ContainerStarted","Data":"683b00116af09f7cd6fbe457a4ae2d9097fa0a6181d698094bb3c2c72535ac81"} Jun 02 20:03:50.043431 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:50.043378 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" podStartSLOduration=1.649558227 podStartE2EDuration="4.043361877s" podCreationTimestamp="2026-06-02 20:03:46 +0000 UTC" firstStartedPulling="2026-06-02 20:03:47.28957158 +0000 UTC m=+514.606784815" lastFinishedPulling="2026-06-02 20:03:49.683375229 +0000 UTC m=+517.000588465" observedRunningTime="2026-06-02 20:03:50.040402392 +0000 UTC m=+517.357615646" watchObservedRunningTime="2026-06-02 20:03:50.043361877 +0000 UTC m=+517.360575131" Jun 02 20:03:50.136799 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:50.136733 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:50.141666 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:50.141639 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:51.018456 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:51.018421 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:03:51.019651 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:03:51.019632 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf" Jun 02 20:04:16.389674 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.389589 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rtz52"] Jun 02 20:04:16.397261 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.397234 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rtz52" Jun 02 20:04:16.401452 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.401419 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 20:04:16.401619 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.401551 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 20:04:16.402404 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.402221 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-kzfqc\"" Jun 02 20:04:16.403416 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.403394 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rtz52"] Jun 02 20:04:16.458468 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.458428 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-759b6\" (UniqueName: \"kubernetes.io/projected/302ddb07-3ec0-482b-9db2-5f47cf7b38f6-kube-api-access-759b6\") pod \"kuadrant-operator-catalog-rtz52\" (UID: \"302ddb07-3ec0-482b-9db2-5f47cf7b38f6\") " pod="kuadrant-system/kuadrant-operator-catalog-rtz52" Jun 02 20:04:16.559028 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.558985 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-759b6\" (UniqueName: \"kubernetes.io/projected/302ddb07-3ec0-482b-9db2-5f47cf7b38f6-kube-api-access-759b6\") pod \"kuadrant-operator-catalog-rtz52\" (UID: \"302ddb07-3ec0-482b-9db2-5f47cf7b38f6\") " pod="kuadrant-system/kuadrant-operator-catalog-rtz52" Jun 02 20:04:16.568060 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.568018 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-759b6\" (UniqueName: \"kubernetes.io/projected/302ddb07-3ec0-482b-9db2-5f47cf7b38f6-kube-api-access-759b6\") pod \"kuadrant-operator-catalog-rtz52\" (UID: \"302ddb07-3ec0-482b-9db2-5f47cf7b38f6\") " pod="kuadrant-system/kuadrant-operator-catalog-rtz52" Jun 02 20:04:16.710109 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.710056 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rtz52" Jun 02 20:04:16.752150 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.752115 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rtz52"] Jun 02 20:04:16.844477 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.844452 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rtz52"] Jun 02 20:04:16.846892 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:04:16.846861 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod302ddb07_3ec0_482b_9db2_5f47cf7b38f6.slice/crio-2d5b64b4f4b946528807f472dd7f6843139f42c360a777d3906402378e8ae01a WatchSource:0}: Error finding container 2d5b64b4f4b946528807f472dd7f6843139f42c360a777d3906402378e8ae01a: Status 404 returned error can't find the container with id 2d5b64b4f4b946528807f472dd7f6843139f42c360a777d3906402378e8ae01a Jun 02 20:04:16.957882 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.957844 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5x7xq"] Jun 02 20:04:16.960893 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.960847 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5x7xq" Jun 02 20:04:16.968397 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:16.968367 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5x7xq"] Jun 02 20:04:17.063921 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:17.063885 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpgrh\" (UniqueName: \"kubernetes.io/projected/a4b29350-a492-4163-8f50-7521395ac15e-kube-api-access-fpgrh\") pod \"kuadrant-operator-catalog-5x7xq\" (UID: \"a4b29350-a492-4163-8f50-7521395ac15e\") " pod="kuadrant-system/kuadrant-operator-catalog-5x7xq" Jun 02 20:04:17.118984 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:17.118946 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rtz52" event={"ID":"302ddb07-3ec0-482b-9db2-5f47cf7b38f6","Type":"ContainerStarted","Data":"2d5b64b4f4b946528807f472dd7f6843139f42c360a777d3906402378e8ae01a"} Jun 02 20:04:17.164553 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:17.164511 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fpgrh\" (UniqueName: \"kubernetes.io/projected/a4b29350-a492-4163-8f50-7521395ac15e-kube-api-access-fpgrh\") pod \"kuadrant-operator-catalog-5x7xq\" (UID: \"a4b29350-a492-4163-8f50-7521395ac15e\") " pod="kuadrant-system/kuadrant-operator-catalog-5x7xq" Jun 02 20:04:17.173992 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:17.173959 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpgrh\" (UniqueName: \"kubernetes.io/projected/a4b29350-a492-4163-8f50-7521395ac15e-kube-api-access-fpgrh\") pod \"kuadrant-operator-catalog-5x7xq\" (UID: \"a4b29350-a492-4163-8f50-7521395ac15e\") " pod="kuadrant-system/kuadrant-operator-catalog-5x7xq" Jun 02 20:04:17.272311 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:17.272216 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5x7xq" Jun 02 20:04:17.420218 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:17.420018 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5x7xq"] Jun 02 20:04:17.422894 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:04:17.422862 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4b29350_a492_4163_8f50_7521395ac15e.slice/crio-19bdb56c11f46023d53def25288c3dd94cb44269ba8681885f403d01704497c0 WatchSource:0}: Error finding container 19bdb56c11f46023d53def25288c3dd94cb44269ba8681885f403d01704497c0: Status 404 returned error can't find the container with id 19bdb56c11f46023d53def25288c3dd94cb44269ba8681885f403d01704497c0 Jun 02 20:04:18.124877 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:18.124833 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5x7xq" event={"ID":"a4b29350-a492-4163-8f50-7521395ac15e","Type":"ContainerStarted","Data":"19bdb56c11f46023d53def25288c3dd94cb44269ba8681885f403d01704497c0"} Jun 02 20:04:20.133406 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:20.133364 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5x7xq" event={"ID":"a4b29350-a492-4163-8f50-7521395ac15e","Type":"ContainerStarted","Data":"5d719932bf76bc4c863ca5a2f43e6f0af0e6140126e75cb3eb8d2965e18fefc5"} Jun 02 20:04:20.134801 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:20.134767 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rtz52" event={"ID":"302ddb07-3ec0-482b-9db2-5f47cf7b38f6","Type":"ContainerStarted","Data":"c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461"} Jun 02 20:04:20.134955 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:20.134811 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-rtz52" podUID="302ddb07-3ec0-482b-9db2-5f47cf7b38f6" containerName="registry-server" containerID="cri-o://c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461" gracePeriod=2 Jun 02 20:04:20.150036 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:20.149946 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5x7xq" podStartSLOduration=1.929884297 podStartE2EDuration="4.149928507s" podCreationTimestamp="2026-06-02 20:04:16 +0000 UTC" firstStartedPulling="2026-06-02 20:04:17.424311742 +0000 UTC m=+544.741524974" lastFinishedPulling="2026-06-02 20:04:19.644355948 +0000 UTC m=+546.961569184" observedRunningTime="2026-06-02 20:04:20.148246615 +0000 UTC m=+547.465459891" watchObservedRunningTime="2026-06-02 20:04:20.149928507 +0000 UTC m=+547.467141757" Jun 02 20:04:20.164786 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:20.164707 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-rtz52" podStartSLOduration=1.4287675530000001 podStartE2EDuration="4.164691678s" podCreationTimestamp="2026-06-02 20:04:16 +0000 UTC" firstStartedPulling="2026-06-02 20:04:16.848338437 +0000 UTC m=+544.165551672" lastFinishedPulling="2026-06-02 20:04:19.584262562 +0000 UTC m=+546.901475797" observedRunningTime="2026-06-02 20:04:20.1643854 +0000 UTC m=+547.481598655" watchObservedRunningTime="2026-06-02 20:04:20.164691678 +0000 UTC m=+547.481904932" Jun 02 20:04:20.383450 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:20.383386 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rtz52" Jun 02 20:04:20.496849 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:20.496809 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-759b6\" (UniqueName: \"kubernetes.io/projected/302ddb07-3ec0-482b-9db2-5f47cf7b38f6-kube-api-access-759b6\") pod \"302ddb07-3ec0-482b-9db2-5f47cf7b38f6\" (UID: \"302ddb07-3ec0-482b-9db2-5f47cf7b38f6\") " Jun 02 20:04:20.499122 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:20.499094 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/302ddb07-3ec0-482b-9db2-5f47cf7b38f6-kube-api-access-759b6" (OuterVolumeSpecName: "kube-api-access-759b6") pod "302ddb07-3ec0-482b-9db2-5f47cf7b38f6" (UID: "302ddb07-3ec0-482b-9db2-5f47cf7b38f6"). InnerVolumeSpecName "kube-api-access-759b6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:04:20.598304 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:20.598264 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-759b6\" (UniqueName: \"kubernetes.io/projected/302ddb07-3ec0-482b-9db2-5f47cf7b38f6-kube-api-access-759b6\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 20:04:21.139489 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:21.139452 2548 generic.go:358] "Generic (PLEG): container finished" podID="302ddb07-3ec0-482b-9db2-5f47cf7b38f6" containerID="c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461" exitCode=0 Jun 02 20:04:21.139971 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:21.139498 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rtz52" event={"ID":"302ddb07-3ec0-482b-9db2-5f47cf7b38f6","Type":"ContainerDied","Data":"c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461"} Jun 02 20:04:21.139971 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:21.139515 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rtz52" Jun 02 20:04:21.139971 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:21.139538 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rtz52" event={"ID":"302ddb07-3ec0-482b-9db2-5f47cf7b38f6","Type":"ContainerDied","Data":"2d5b64b4f4b946528807f472dd7f6843139f42c360a777d3906402378e8ae01a"} Jun 02 20:04:21.139971 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:21.139556 2548 scope.go:117] "RemoveContainer" containerID="c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461" Jun 02 20:04:21.148275 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:21.148258 2548 scope.go:117] "RemoveContainer" containerID="c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461" Jun 02 20:04:21.148534 ip-10-0-129-157 kubenswrapper[2548]: E0602 20:04:21.148516 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461\": container with ID starting with c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461 not found: ID does not exist" containerID="c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461" Jun 02 20:04:21.148582 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:21.148544 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461"} err="failed to get container status \"c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461\": rpc error: code = NotFound desc = could not find container \"c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461\": container with ID starting with c98282a9f68c90e06d5c869c398fa1a22d8e0a61d881eab2cd37555a7cfa5461 not found: ID does not exist" Jun 02 20:04:21.162308 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:21.162272 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rtz52"] Jun 02 20:04:21.166501 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:21.166475 2548 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rtz52"] Jun 02 20:04:21.318919 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:21.318886 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="302ddb07-3ec0-482b-9db2-5f47cf7b38f6" path="/var/lib/kubelet/pods/302ddb07-3ec0-482b-9db2-5f47cf7b38f6/volumes" Jun 02 20:04:27.273245 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:27.273205 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-5x7xq" Jun 02 20:04:27.273245 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:27.273246 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-5x7xq" Jun 02 20:04:27.296175 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:27.296144 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-5x7xq" Jun 02 20:04:28.186690 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:28.186662 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-5x7xq" Jun 02 20:04:48.777624 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:48.777580 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-m9r26"] Jun 02 20:04:48.778155 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:48.778043 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="302ddb07-3ec0-482b-9db2-5f47cf7b38f6" containerName="registry-server" Jun 02 20:04:48.778155 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:48.778057 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="302ddb07-3ec0-482b-9db2-5f47cf7b38f6" containerName="registry-server" Jun 02 20:04:48.778155 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:48.778131 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="302ddb07-3ec0-482b-9db2-5f47cf7b38f6" containerName="registry-server" Jun 02 20:04:48.780245 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:48.780228 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-m9r26" Jun 02 20:04:48.787201 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:48.787159 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-wcbps\"" Jun 02 20:04:48.792608 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:48.792571 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-m9r26"] Jun 02 20:04:48.868212 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:48.868170 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcr7h\" (UniqueName: \"kubernetes.io/projected/5d0a7299-4f6c-4900-8245-23223f2cbc12-kube-api-access-mcr7h\") pod \"authorino-operator-657f44b778-m9r26\" (UID: \"5d0a7299-4f6c-4900-8245-23223f2cbc12\") " pod="kuadrant-system/authorino-operator-657f44b778-m9r26" Jun 02 20:04:48.969303 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:48.969260 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mcr7h\" (UniqueName: \"kubernetes.io/projected/5d0a7299-4f6c-4900-8245-23223f2cbc12-kube-api-access-mcr7h\") pod \"authorino-operator-657f44b778-m9r26\" (UID: \"5d0a7299-4f6c-4900-8245-23223f2cbc12\") " pod="kuadrant-system/authorino-operator-657f44b778-m9r26" Jun 02 20:04:48.983592 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:48.983556 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcr7h\" (UniqueName: \"kubernetes.io/projected/5d0a7299-4f6c-4900-8245-23223f2cbc12-kube-api-access-mcr7h\") pod \"authorino-operator-657f44b778-m9r26\" (UID: \"5d0a7299-4f6c-4900-8245-23223f2cbc12\") " pod="kuadrant-system/authorino-operator-657f44b778-m9r26" Jun 02 20:04:49.091415 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:49.091318 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-m9r26" Jun 02 20:04:49.233023 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:49.232992 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-m9r26"] Jun 02 20:04:49.245003 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:04:49.244969 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d0a7299_4f6c_4900_8245_23223f2cbc12.slice/crio-a2abea5055f2bb3756626c10ef87041077e54a2331b333f68dbd3538da9d0789 WatchSource:0}: Error finding container a2abea5055f2bb3756626c10ef87041077e54a2331b333f68dbd3538da9d0789: Status 404 returned error can't find the container with id a2abea5055f2bb3756626c10ef87041077e54a2331b333f68dbd3538da9d0789 Jun 02 20:04:50.247864 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:50.247826 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-m9r26" event={"ID":"5d0a7299-4f6c-4900-8245-23223f2cbc12","Type":"ContainerStarted","Data":"a2abea5055f2bb3756626c10ef87041077e54a2331b333f68dbd3538da9d0789"} Jun 02 20:04:52.230223 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.230179 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf"] Jun 02 20:04:52.253430 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.253397 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf"] Jun 02 20:04:52.253720 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.253692 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf" Jun 02 20:04:52.259699 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.259671 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-m9r26" event={"ID":"5d0a7299-4f6c-4900-8245-23223f2cbc12","Type":"ContainerStarted","Data":"15a99610e3c39654e21de8d69ac7af580d074c6ab419b2919ada84012e053424"} Jun 02 20:04:52.259860 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.259788 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-m9r26" Jun 02 20:04:52.261977 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.261956 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 02 20:04:52.262086 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.261980 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dj2bj\"" Jun 02 20:04:52.299707 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.299643 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-m9r26" podStartSLOduration=2.122995958 podStartE2EDuration="4.299622042s" podCreationTimestamp="2026-06-02 20:04:48 +0000 UTC" firstStartedPulling="2026-06-02 20:04:49.247459876 +0000 UTC m=+576.564673108" lastFinishedPulling="2026-06-02 20:04:51.424085751 +0000 UTC m=+578.741299192" observedRunningTime="2026-06-02 20:04:52.298425176 +0000 UTC m=+579.615638429" watchObservedRunningTime="2026-06-02 20:04:52.299622042 +0000 UTC m=+579.616835297" Jun 02 20:04:52.300064 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.300040 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l488\" (UniqueName: \"kubernetes.io/projected/c27f5c97-f179-4ad9-acbb-24a12b45fa13-kube-api-access-6l488\") pod \"dns-operator-controller-manager-648d5c98bc-tzrtf\" (UID: \"c27f5c97-f179-4ad9-acbb-24a12b45fa13\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf" Jun 02 20:04:52.401082 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.401035 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6l488\" (UniqueName: \"kubernetes.io/projected/c27f5c97-f179-4ad9-acbb-24a12b45fa13-kube-api-access-6l488\") pod \"dns-operator-controller-manager-648d5c98bc-tzrtf\" (UID: \"c27f5c97-f179-4ad9-acbb-24a12b45fa13\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf" Jun 02 20:04:52.411432 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.411392 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l488\" (UniqueName: \"kubernetes.io/projected/c27f5c97-f179-4ad9-acbb-24a12b45fa13-kube-api-access-6l488\") pod \"dns-operator-controller-manager-648d5c98bc-tzrtf\" (UID: \"c27f5c97-f179-4ad9-acbb-24a12b45fa13\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf" Jun 02 20:04:52.566944 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.566851 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf" Jun 02 20:04:52.713425 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:52.713397 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf"] Jun 02 20:04:52.716123 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:04:52.716092 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc27f5c97_f179_4ad9_acbb_24a12b45fa13.slice/crio-39d30d2e7b98773590294f095d26289bb41bd7a72e4f2800bb2cfdad765334b2 WatchSource:0}: Error finding container 39d30d2e7b98773590294f095d26289bb41bd7a72e4f2800bb2cfdad765334b2: Status 404 returned error can't find the container with id 39d30d2e7b98773590294f095d26289bb41bd7a72e4f2800bb2cfdad765334b2 Jun 02 20:04:53.264273 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:53.264235 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf" event={"ID":"c27f5c97-f179-4ad9-acbb-24a12b45fa13","Type":"ContainerStarted","Data":"39d30d2e7b98773590294f095d26289bb41bd7a72e4f2800bb2cfdad765334b2"} Jun 02 20:04:55.048990 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:55.048952 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4"] Jun 02 20:04:55.053751 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:55.053719 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" Jun 02 20:04:55.058211 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:55.058186 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-cw9p4\"" Jun 02 20:04:55.084502 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:55.084471 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4"] Jun 02 20:04:55.126562 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:55.126517 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg5lt\" (UniqueName: \"kubernetes.io/projected/47d465c4-9be4-4a16-b20c-d93f28bf04c4-kube-api-access-zg5lt\") pod \"limitador-operator-controller-manager-85c4996f8c-wtzx4\" (UID: \"47d465c4-9be4-4a16-b20c-d93f28bf04c4\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" Jun 02 20:04:55.227567 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:55.227527 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zg5lt\" (UniqueName: \"kubernetes.io/projected/47d465c4-9be4-4a16-b20c-d93f28bf04c4-kube-api-access-zg5lt\") pod \"limitador-operator-controller-manager-85c4996f8c-wtzx4\" (UID: \"47d465c4-9be4-4a16-b20c-d93f28bf04c4\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" Jun 02 20:04:55.244050 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:55.244014 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg5lt\" (UniqueName: \"kubernetes.io/projected/47d465c4-9be4-4a16-b20c-d93f28bf04c4-kube-api-access-zg5lt\") pod \"limitador-operator-controller-manager-85c4996f8c-wtzx4\" (UID: \"47d465c4-9be4-4a16-b20c-d93f28bf04c4\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" Jun 02 20:04:55.276316 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:55.276222 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf" event={"ID":"c27f5c97-f179-4ad9-acbb-24a12b45fa13","Type":"ContainerStarted","Data":"a1b8c12869bbbf966079bc8af23c883511686bdd418f99da4d9c7fda79499336"} Jun 02 20:04:55.276471 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:55.276376 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf" Jun 02 20:04:55.307274 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:55.307217 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf" podStartSLOduration=1.004285433 podStartE2EDuration="3.307200246s" podCreationTimestamp="2026-06-02 20:04:52 +0000 UTC" firstStartedPulling="2026-06-02 20:04:52.71815206 +0000 UTC m=+580.035365292" lastFinishedPulling="2026-06-02 20:04:55.021066873 +0000 UTC m=+582.338280105" observedRunningTime="2026-06-02 20:04:55.306284741 +0000 UTC m=+582.623497997" watchObservedRunningTime="2026-06-02 20:04:55.307200246 +0000 UTC m=+582.624413499" Jun 02 20:04:55.373963 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:55.373911 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" Jun 02 20:04:55.533608 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:55.533578 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4"] Jun 02 20:04:55.536090 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:04:55.536056 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d465c4_9be4_4a16_b20c_d93f28bf04c4.slice/crio-60c805db6556208720feb4b3ed9be35abcdec6647a8e2f7fe1bbbfe3147d5b27 WatchSource:0}: Error finding container 60c805db6556208720feb4b3ed9be35abcdec6647a8e2f7fe1bbbfe3147d5b27: Status 404 returned error can't find the container with id 60c805db6556208720feb4b3ed9be35abcdec6647a8e2f7fe1bbbfe3147d5b27 Jun 02 20:04:56.281608 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:56.281573 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" event={"ID":"47d465c4-9be4-4a16-b20c-d93f28bf04c4","Type":"ContainerStarted","Data":"60c805db6556208720feb4b3ed9be35abcdec6647a8e2f7fe1bbbfe3147d5b27"} Jun 02 20:04:58.290238 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:58.290141 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" event={"ID":"47d465c4-9be4-4a16-b20c-d93f28bf04c4","Type":"ContainerStarted","Data":"929258d50a323a290b7972a8a6876b303a11f0d8f78246feb91f8a68752f33db"} Jun 02 20:04:58.290622 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:58.290250 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" Jun 02 20:04:58.315032 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:04:58.314960 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" podStartSLOduration=0.854161227 podStartE2EDuration="3.314944822s" podCreationTimestamp="2026-06-02 20:04:55 +0000 UTC" firstStartedPulling="2026-06-02 20:04:55.538173379 +0000 UTC m=+582.855386612" lastFinishedPulling="2026-06-02 20:04:57.998956976 +0000 UTC m=+585.316170207" observedRunningTime="2026-06-02 20:04:58.313645181 +0000 UTC m=+585.630858438" watchObservedRunningTime="2026-06-02 20:04:58.314944822 +0000 UTC m=+585.632158075" Jun 02 20:05:03.266663 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:03.266631 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-m9r26" Jun 02 20:05:06.284398 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:06.284365 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-tzrtf" Jun 02 20:05:09.296272 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:09.296239 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" Jun 02 20:05:13.167154 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:13.167124 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:05:13.167540 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:13.167427 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:05:19.020688 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.020646 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn"] Jun 02 20:05:19.025258 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.025236 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" Jun 02 20:05:19.028643 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.028619 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-cdrmv\"" Jun 02 20:05:19.041257 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.041223 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn"] Jun 02 20:05:19.166790 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.166734 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r44b2\" (UniqueName: \"kubernetes.io/projected/0b8ae75f-dd67-43dc-9d8e-179484553190-kube-api-access-r44b2\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-glkcn\" (UID: \"0b8ae75f-dd67-43dc-9d8e-179484553190\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" Jun 02 20:05:19.166954 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.166880 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b8ae75f-dd67-43dc-9d8e-179484553190-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-glkcn\" (UID: \"0b8ae75f-dd67-43dc-9d8e-179484553190\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" Jun 02 20:05:19.268140 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.268092 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r44b2\" (UniqueName: \"kubernetes.io/projected/0b8ae75f-dd67-43dc-9d8e-179484553190-kube-api-access-r44b2\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-glkcn\" (UID: \"0b8ae75f-dd67-43dc-9d8e-179484553190\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" Jun 02 20:05:19.268312 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.268195 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b8ae75f-dd67-43dc-9d8e-179484553190-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-glkcn\" (UID: \"0b8ae75f-dd67-43dc-9d8e-179484553190\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" Jun 02 20:05:19.268520 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.268504 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b8ae75f-dd67-43dc-9d8e-179484553190-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-glkcn\" (UID: \"0b8ae75f-dd67-43dc-9d8e-179484553190\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" Jun 02 20:05:19.278588 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.278520 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r44b2\" (UniqueName: \"kubernetes.io/projected/0b8ae75f-dd67-43dc-9d8e-179484553190-kube-api-access-r44b2\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-glkcn\" (UID: \"0b8ae75f-dd67-43dc-9d8e-179484553190\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" Jun 02 20:05:19.335507 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.335461 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" Jun 02 20:05:19.503826 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.495996 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn"] Jun 02 20:05:19.509465 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:05:19.509421 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8ae75f_dd67_43dc_9d8e_179484553190.slice/crio-b69ce17c4a0d66c6ae39b922cfaa187e5db873acfe0f903c5dadb5a8d09e07f6 WatchSource:0}: Error finding container b69ce17c4a0d66c6ae39b922cfaa187e5db873acfe0f903c5dadb5a8d09e07f6: Status 404 returned error can't find the container with id b69ce17c4a0d66c6ae39b922cfaa187e5db873acfe0f903c5dadb5a8d09e07f6 Jun 02 20:05:19.677016 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.676977 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn"] Jun 02 20:05:19.684043 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.684007 2548 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn"] Jun 02 20:05:19.699202 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.699168 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4"] Jun 02 20:05:19.699482 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.699445 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" podUID="47d465c4-9be4-4a16-b20c-d93f28bf04c4" containerName="manager" containerID="cri-o://929258d50a323a290b7972a8a6876b303a11f0d8f78246feb91f8a68752f33db" gracePeriod=2 Jun 02 20:05:19.713799 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.711612 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st"] Jun 02 20:05:19.715712 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.715683 2548 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4"] Jun 02 20:05:19.715871 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.715850 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" Jun 02 20:05:19.733712 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.733674 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb"] Jun 02 20:05:19.734201 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.734181 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47d465c4-9be4-4a16-b20c-d93f28bf04c4" containerName="manager" Jun 02 20:05:19.734201 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.734202 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="47d465c4-9be4-4a16-b20c-d93f28bf04c4" containerName="manager" Jun 02 20:05:19.734349 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.734293 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="47d465c4-9be4-4a16-b20c-d93f28bf04c4" containerName="manager" Jun 02 20:05:19.737881 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.737854 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb" Jun 02 20:05:19.739411 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.739386 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st"] Jun 02 20:05:19.757629 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.757561 2548 status_manager.go:895] "Failed to get status for pod" podUID="47d465c4-9be4-4a16-b20c-d93f28bf04c4" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" err="pods \"limitador-operator-controller-manager-85c4996f8c-wtzx4\" is forbidden: User \"system:node:ip-10-0-129-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-157.ec2.internal' and this object" Jun 02 20:05:19.759290 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.759256 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb"] Jun 02 20:05:19.760256 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.760222 2548 status_manager.go:895] "Failed to get status for pod" podUID="47d465c4-9be4-4a16-b20c-d93f28bf04c4" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" err="pods \"limitador-operator-controller-manager-85c4996f8c-wtzx4\" is forbidden: User \"system:node:ip-10-0-129-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-157.ec2.internal' and this object" Jun 02 20:05:19.876357 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.876313 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffpth\" (UniqueName: \"kubernetes.io/projected/8d93ac9c-93d5-4d6b-ba6a-2017ce6f4a8d-kube-api-access-ffpth\") pod \"limitador-operator-controller-manager-85c4996f8c-nrpkb\" (UID: \"8d93ac9c-93d5-4d6b-ba6a-2017ce6f4a8d\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb" Jun 02 20:05:19.876504 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.876378 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac5f7f69-0c17-48d1-be6e-3e53521ceb38-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-jb2st\" (UID: \"ac5f7f69-0c17-48d1-be6e-3e53521ceb38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" Jun 02 20:05:19.876504 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.876445 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8zss\" (UniqueName: \"kubernetes.io/projected/ac5f7f69-0c17-48d1-be6e-3e53521ceb38-kube-api-access-h8zss\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-jb2st\" (UID: \"ac5f7f69-0c17-48d1-be6e-3e53521ceb38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" Jun 02 20:05:19.935103 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.935068 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" Jun 02 20:05:19.937456 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.937423 2548 status_manager.go:895] "Failed to get status for pod" podUID="47d465c4-9be4-4a16-b20c-d93f28bf04c4" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" err="pods \"limitador-operator-controller-manager-85c4996f8c-wtzx4\" is forbidden: User \"system:node:ip-10-0-129-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-157.ec2.internal' and this object" Jun 02 20:05:19.978013 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.977971 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac5f7f69-0c17-48d1-be6e-3e53521ceb38-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-jb2st\" (UID: \"ac5f7f69-0c17-48d1-be6e-3e53521ceb38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" Jun 02 20:05:19.978199 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.978046 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h8zss\" (UniqueName: \"kubernetes.io/projected/ac5f7f69-0c17-48d1-be6e-3e53521ceb38-kube-api-access-h8zss\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-jb2st\" (UID: \"ac5f7f69-0c17-48d1-be6e-3e53521ceb38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" Jun 02 20:05:19.978199 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.978179 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ffpth\" (UniqueName: \"kubernetes.io/projected/8d93ac9c-93d5-4d6b-ba6a-2017ce6f4a8d-kube-api-access-ffpth\") pod \"limitador-operator-controller-manager-85c4996f8c-nrpkb\" (UID: \"8d93ac9c-93d5-4d6b-ba6a-2017ce6f4a8d\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb" Jun 02 20:05:19.978468 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.978448 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac5f7f69-0c17-48d1-be6e-3e53521ceb38-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-jb2st\" (UID: \"ac5f7f69-0c17-48d1-be6e-3e53521ceb38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" Jun 02 20:05:19.988214 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.988172 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffpth\" (UniqueName: \"kubernetes.io/projected/8d93ac9c-93d5-4d6b-ba6a-2017ce6f4a8d-kube-api-access-ffpth\") pod \"limitador-operator-controller-manager-85c4996f8c-nrpkb\" (UID: \"8d93ac9c-93d5-4d6b-ba6a-2017ce6f4a8d\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb" Jun 02 20:05:19.988214 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:19.988179 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8zss\" (UniqueName: \"kubernetes.io/projected/ac5f7f69-0c17-48d1-be6e-3e53521ceb38-kube-api-access-h8zss\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-jb2st\" (UID: \"ac5f7f69-0c17-48d1-be6e-3e53521ceb38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" Jun 02 20:05:20.079002 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.078965 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg5lt\" (UniqueName: \"kubernetes.io/projected/47d465c4-9be4-4a16-b20c-d93f28bf04c4-kube-api-access-zg5lt\") pod \"47d465c4-9be4-4a16-b20c-d93f28bf04c4\" (UID: \"47d465c4-9be4-4a16-b20c-d93f28bf04c4\") " Jun 02 20:05:20.081009 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.080983 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47d465c4-9be4-4a16-b20c-d93f28bf04c4-kube-api-access-zg5lt" (OuterVolumeSpecName: "kube-api-access-zg5lt") pod "47d465c4-9be4-4a16-b20c-d93f28bf04c4" (UID: "47d465c4-9be4-4a16-b20c-d93f28bf04c4"). InnerVolumeSpecName "kube-api-access-zg5lt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:05:20.083192 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.083171 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" Jun 02 20:05:20.092068 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.092000 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb" Jun 02 20:05:20.180661 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.180579 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg5lt\" (UniqueName: \"kubernetes.io/projected/47d465c4-9be4-4a16-b20c-d93f28bf04c4-kube-api-access-zg5lt\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 20:05:20.249117 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.248294 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st"] Jun 02 20:05:20.254154 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:05:20.254122 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5f7f69_0c17_48d1_be6e_3e53521ceb38.slice/crio-0d2270a2019b7afef76e1c49b37d01dd39b2176fa80babebea2c7d8d8f50ee95 WatchSource:0}: Error finding container 0d2270a2019b7afef76e1c49b37d01dd39b2176fa80babebea2c7d8d8f50ee95: Status 404 returned error can't find the container with id 0d2270a2019b7afef76e1c49b37d01dd39b2176fa80babebea2c7d8d8f50ee95 Jun 02 20:05:20.266482 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.266452 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb"] Jun 02 20:05:20.268692 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:05:20.268663 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d93ac9c_93d5_4d6b_ba6a_2017ce6f4a8d.slice/crio-4e9ffe0dcba8685387fac05dd1953b6d06f19b608bed2388d09fa87da9b92908 WatchSource:0}: Error finding container 4e9ffe0dcba8685387fac05dd1953b6d06f19b608bed2388d09fa87da9b92908: Status 404 returned error can't find the container with id 4e9ffe0dcba8685387fac05dd1953b6d06f19b608bed2388d09fa87da9b92908 Jun 02 20:05:20.381203 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.381169 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb" event={"ID":"8d93ac9c-93d5-4d6b-ba6a-2017ce6f4a8d","Type":"ContainerStarted","Data":"f1060a4d349ea0ae7f8dd77d1d5778046abfad85da9c6dbd17ea544d6f147759"} Jun 02 20:05:20.381203 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.381205 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb" event={"ID":"8d93ac9c-93d5-4d6b-ba6a-2017ce6f4a8d","Type":"ContainerStarted","Data":"4e9ffe0dcba8685387fac05dd1953b6d06f19b608bed2388d09fa87da9b92908"} Jun 02 20:05:20.381459 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.381268 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb" Jun 02 20:05:20.382406 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.382380 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" event={"ID":"ac5f7f69-0c17-48d1-be6e-3e53521ceb38","Type":"ContainerStarted","Data":"0d2270a2019b7afef76e1c49b37d01dd39b2176fa80babebea2c7d8d8f50ee95"} Jun 02 20:05:20.383445 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.383415 2548 status_manager.go:895] "Failed to get status for pod" podUID="47d465c4-9be4-4a16-b20c-d93f28bf04c4" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" err="pods \"limitador-operator-controller-manager-85c4996f8c-wtzx4\" is forbidden: User \"system:node:ip-10-0-129-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-157.ec2.internal' and this object" Jun 02 20:05:20.383544 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.383471 2548 generic.go:358] "Generic (PLEG): container finished" podID="47d465c4-9be4-4a16-b20c-d93f28bf04c4" containerID="929258d50a323a290b7972a8a6876b303a11f0d8f78246feb91f8a68752f33db" exitCode=0 Jun 02 20:05:20.383544 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.383524 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" Jun 02 20:05:20.383650 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.383553 2548 scope.go:117] "RemoveContainer" containerID="929258d50a323a290b7972a8a6876b303a11f0d8f78246feb91f8a68752f33db" Jun 02 20:05:20.392462 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.392441 2548 scope.go:117] "RemoveContainer" containerID="929258d50a323a290b7972a8a6876b303a11f0d8f78246feb91f8a68752f33db" Jun 02 20:05:20.392732 ip-10-0-129-157 kubenswrapper[2548]: E0602 20:05:20.392712 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"929258d50a323a290b7972a8a6876b303a11f0d8f78246feb91f8a68752f33db\": container with ID starting with 929258d50a323a290b7972a8a6876b303a11f0d8f78246feb91f8a68752f33db not found: ID does not exist" containerID="929258d50a323a290b7972a8a6876b303a11f0d8f78246feb91f8a68752f33db" Jun 02 20:05:20.392796 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.392768 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"929258d50a323a290b7972a8a6876b303a11f0d8f78246feb91f8a68752f33db"} err="failed to get container status \"929258d50a323a290b7972a8a6876b303a11f0d8f78246feb91f8a68752f33db\": rpc error: code = NotFound desc = could not find container \"929258d50a323a290b7972a8a6876b303a11f0d8f78246feb91f8a68752f33db\": container with ID starting with 929258d50a323a290b7972a8a6876b303a11f0d8f78246feb91f8a68752f33db not found: ID does not exist" Jun 02 20:05:20.405137 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.405091 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb" podStartSLOduration=1.405075033 podStartE2EDuration="1.405075033s" podCreationTimestamp="2026-06-02 20:05:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 20:05:20.404253312 +0000 UTC m=+607.721466564" watchObservedRunningTime="2026-06-02 20:05:20.405075033 +0000 UTC m=+607.722288286" Jun 02 20:05:20.406761 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:20.406684 2548 status_manager.go:895] "Failed to get status for pod" podUID="47d465c4-9be4-4a16-b20c-d93f28bf04c4" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wtzx4" err="pods \"limitador-operator-controller-manager-85c4996f8c-wtzx4\" is forbidden: User \"system:node:ip-10-0-129-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-157.ec2.internal' and this object" Jun 02 20:05:21.321061 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:21.321026 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47d465c4-9be4-4a16-b20c-d93f28bf04c4" path="/var/lib/kubelet/pods/47d465c4-9be4-4a16-b20c-d93f28bf04c4/volumes" Jun 02 20:05:24.409367 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.409324 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" event={"ID":"ac5f7f69-0c17-48d1-be6e-3e53521ceb38","Type":"ContainerStarted","Data":"4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50"} Jun 02 20:05:24.409850 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.409448 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" Jun 02 20:05:24.410851 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.410809 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" podUID="0b8ae75f-dd67-43dc-9d8e-179484553190" containerName="manager" containerID="cri-o://a252d47a17ab41949973a2eb1c1b465de4c4346b4d7719e2752f79f29e46bccd" gracePeriod=2 Jun 02 20:05:24.429914 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.429857 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" podStartSLOduration=1.935892698 podStartE2EDuration="5.429840604s" podCreationTimestamp="2026-06-02 20:05:19 +0000 UTC" firstStartedPulling="2026-06-02 20:05:20.257160503 +0000 UTC m=+607.574373736" lastFinishedPulling="2026-06-02 20:05:23.751108407 +0000 UTC m=+611.068321642" observedRunningTime="2026-06-02 20:05:24.427902241 +0000 UTC m=+611.745115497" watchObservedRunningTime="2026-06-02 20:05:24.429840604 +0000 UTC m=+611.747053859" Jun 02 20:05:24.430342 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.430314 2548 status_manager.go:895] "Failed to get status for pod" podUID="0b8ae75f-dd67-43dc-9d8e-179484553190" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-glkcn\" is forbidden: User \"system:node:ip-10-0-129-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-157.ec2.internal' and this object" Jun 02 20:05:24.656341 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.656315 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" Jun 02 20:05:24.658607 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.658575 2548 status_manager.go:895] "Failed to get status for pod" podUID="0b8ae75f-dd67-43dc-9d8e-179484553190" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-glkcn\" is forbidden: User \"system:node:ip-10-0-129-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-157.ec2.internal' and this object" Jun 02 20:05:24.727309 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.727209 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b8ae75f-dd67-43dc-9d8e-179484553190-extensions-socket-volume\") pod \"0b8ae75f-dd67-43dc-9d8e-179484553190\" (UID: \"0b8ae75f-dd67-43dc-9d8e-179484553190\") " Jun 02 20:05:24.727309 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.727261 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r44b2\" (UniqueName: \"kubernetes.io/projected/0b8ae75f-dd67-43dc-9d8e-179484553190-kube-api-access-r44b2\") pod \"0b8ae75f-dd67-43dc-9d8e-179484553190\" (UID: \"0b8ae75f-dd67-43dc-9d8e-179484553190\") " Jun 02 20:05:24.727567 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.727535 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b8ae75f-dd67-43dc-9d8e-179484553190-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0b8ae75f-dd67-43dc-9d8e-179484553190" (UID: "0b8ae75f-dd67-43dc-9d8e-179484553190"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 20:05:24.729370 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.729335 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b8ae75f-dd67-43dc-9d8e-179484553190-kube-api-access-r44b2" (OuterVolumeSpecName: "kube-api-access-r44b2") pod "0b8ae75f-dd67-43dc-9d8e-179484553190" (UID: "0b8ae75f-dd67-43dc-9d8e-179484553190"). InnerVolumeSpecName "kube-api-access-r44b2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:05:24.828025 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.827966 2548 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b8ae75f-dd67-43dc-9d8e-179484553190-extensions-socket-volume\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 20:05:24.828025 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:24.828020 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r44b2\" (UniqueName: \"kubernetes.io/projected/0b8ae75f-dd67-43dc-9d8e-179484553190-kube-api-access-r44b2\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 20:05:25.318892 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:25.318858 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b8ae75f-dd67-43dc-9d8e-179484553190" path="/var/lib/kubelet/pods/0b8ae75f-dd67-43dc-9d8e-179484553190/volumes" Jun 02 20:05:25.415735 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:25.415692 2548 generic.go:358] "Generic (PLEG): container finished" podID="0b8ae75f-dd67-43dc-9d8e-179484553190" containerID="a252d47a17ab41949973a2eb1c1b465de4c4346b4d7719e2752f79f29e46bccd" exitCode=2 Jun 02 20:05:25.416214 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:25.415769 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" Jun 02 20:05:25.416214 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:25.415787 2548 scope.go:117] "RemoveContainer" containerID="a252d47a17ab41949973a2eb1c1b465de4c4346b4d7719e2752f79f29e46bccd" Jun 02 20:05:25.418316 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:25.418279 2548 status_manager.go:895] "Failed to get status for pod" podUID="0b8ae75f-dd67-43dc-9d8e-179484553190" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-glkcn\" is forbidden: User \"system:node:ip-10-0-129-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-157.ec2.internal' and this object" Jun 02 20:05:25.420169 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:25.420144 2548 status_manager.go:895] "Failed to get status for pod" podUID="0b8ae75f-dd67-43dc-9d8e-179484553190" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-glkcn" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-glkcn\" is forbidden: User \"system:node:ip-10-0-129-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-157.ec2.internal' and this object" Jun 02 20:05:25.424485 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:25.424465 2548 scope.go:117] "RemoveContainer" containerID="a252d47a17ab41949973a2eb1c1b465de4c4346b4d7719e2752f79f29e46bccd" Jun 02 20:05:25.425005 ip-10-0-129-157 kubenswrapper[2548]: E0602 20:05:25.424986 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a252d47a17ab41949973a2eb1c1b465de4c4346b4d7719e2752f79f29e46bccd\": container with ID starting with a252d47a17ab41949973a2eb1c1b465de4c4346b4d7719e2752f79f29e46bccd not found: ID does not exist" containerID="a252d47a17ab41949973a2eb1c1b465de4c4346b4d7719e2752f79f29e46bccd" Jun 02 20:05:25.425059 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:25.425016 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a252d47a17ab41949973a2eb1c1b465de4c4346b4d7719e2752f79f29e46bccd"} err="failed to get container status \"a252d47a17ab41949973a2eb1c1b465de4c4346b4d7719e2752f79f29e46bccd\": rpc error: code = NotFound desc = could not find container \"a252d47a17ab41949973a2eb1c1b465de4c4346b4d7719e2752f79f29e46bccd\": container with ID starting with a252d47a17ab41949973a2eb1c1b465de4c4346b4d7719e2752f79f29e46bccd not found: ID does not exist" Jun 02 20:05:31.394066 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:31.393979 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-nrpkb" Jun 02 20:05:35.418481 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:35.418449 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" Jun 02 20:05:48.082072 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.082033 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st"] Jun 02 20:05:48.082506 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.082275 2548 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" podUID="ac5f7f69-0c17-48d1-be6e-3e53521ceb38" containerName="manager" containerID="cri-o://4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50" gracePeriod=10 Jun 02 20:05:48.336678 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.336605 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" Jun 02 20:05:48.456952 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.456916 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8zss\" (UniqueName: \"kubernetes.io/projected/ac5f7f69-0c17-48d1-be6e-3e53521ceb38-kube-api-access-h8zss\") pod \"ac5f7f69-0c17-48d1-be6e-3e53521ceb38\" (UID: \"ac5f7f69-0c17-48d1-be6e-3e53521ceb38\") " Jun 02 20:05:48.456952 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.456966 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac5f7f69-0c17-48d1-be6e-3e53521ceb38-extensions-socket-volume\") pod \"ac5f7f69-0c17-48d1-be6e-3e53521ceb38\" (UID: \"ac5f7f69-0c17-48d1-be6e-3e53521ceb38\") " Jun 02 20:05:48.457627 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.457596 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5f7f69-0c17-48d1-be6e-3e53521ceb38-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ac5f7f69-0c17-48d1-be6e-3e53521ceb38" (UID: "ac5f7f69-0c17-48d1-be6e-3e53521ceb38"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 20:05:48.459156 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.459135 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac5f7f69-0c17-48d1-be6e-3e53521ceb38-kube-api-access-h8zss" (OuterVolumeSpecName: "kube-api-access-h8zss") pod "ac5f7f69-0c17-48d1-be6e-3e53521ceb38" (UID: "ac5f7f69-0c17-48d1-be6e-3e53521ceb38"). InnerVolumeSpecName "kube-api-access-h8zss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:05:48.506450 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.506416 2548 generic.go:358] "Generic (PLEG): container finished" podID="ac5f7f69-0c17-48d1-be6e-3e53521ceb38" containerID="4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50" exitCode=0 Jun 02 20:05:48.506640 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.506468 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" Jun 02 20:05:48.506640 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.506504 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" event={"ID":"ac5f7f69-0c17-48d1-be6e-3e53521ceb38","Type":"ContainerDied","Data":"4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50"} Jun 02 20:05:48.506640 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.506555 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st" event={"ID":"ac5f7f69-0c17-48d1-be6e-3e53521ceb38","Type":"ContainerDied","Data":"0d2270a2019b7afef76e1c49b37d01dd39b2176fa80babebea2c7d8d8f50ee95"} Jun 02 20:05:48.506640 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.506573 2548 scope.go:117] "RemoveContainer" containerID="4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50" Jun 02 20:05:48.515510 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.515490 2548 scope.go:117] "RemoveContainer" containerID="4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50" Jun 02 20:05:48.515792 ip-10-0-129-157 kubenswrapper[2548]: E0602 20:05:48.515773 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50\": container with ID starting with 4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50 not found: ID does not exist" containerID="4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50" Jun 02 20:05:48.515981 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.515802 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50"} err="failed to get container status \"4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50\": rpc error: code = NotFound desc = could not find container \"4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50\": container with ID starting with 4dd3a0a243c19f4412a54d11d4ea9cd7b755cfee1348de476420e0cc26f32a50 not found: ID does not exist" Jun 02 20:05:48.532340 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.532304 2548 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st"] Jun 02 20:05:48.535494 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.535469 2548 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-jb2st"] Jun 02 20:05:48.558237 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.558202 2548 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac5f7f69-0c17-48d1-be6e-3e53521ceb38-extensions-socket-volume\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 20:05:48.558237 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:48.558237 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8zss\" (UniqueName: \"kubernetes.io/projected/ac5f7f69-0c17-48d1-be6e-3e53521ceb38-kube-api-access-h8zss\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 20:05:49.318698 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:49.318660 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac5f7f69-0c17-48d1-be6e-3e53521ceb38" path="/var/lib/kubelet/pods/ac5f7f69-0c17-48d1-be6e-3e53521ceb38/volumes" Jun 02 20:05:51.820564 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.820531 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc"] Jun 02 20:05:51.821037 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.820964 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac5f7f69-0c17-48d1-be6e-3e53521ceb38" containerName="manager" Jun 02 20:05:51.821037 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.820976 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac5f7f69-0c17-48d1-be6e-3e53521ceb38" containerName="manager" Jun 02 20:05:51.821037 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.820993 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b8ae75f-dd67-43dc-9d8e-179484553190" containerName="manager" Jun 02 20:05:51.821037 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.820998 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b8ae75f-dd67-43dc-9d8e-179484553190" containerName="manager" Jun 02 20:05:51.821172 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.821057 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="ac5f7f69-0c17-48d1-be6e-3e53521ceb38" containerName="manager" Jun 02 20:05:51.821172 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.821066 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="0b8ae75f-dd67-43dc-9d8e-179484553190" containerName="manager" Jun 02 20:05:51.824190 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.824161 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:51.827455 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.827424 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-986mz\"" Jun 02 20:05:51.837462 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.837431 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc"] Jun 02 20:05:51.994814 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.994736 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:51.994814 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.994815 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:51.995053 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.994845 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:51.995053 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.994879 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:51.995053 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.994956 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91a59811-a055-4e42-bc2f-beef27d8fa1d-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:51.995053 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.994994 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91a59811-a055-4e42-bc2f-beef27d8fa1d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:51.995053 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.995028 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91a59811-a055-4e42-bc2f-beef27d8fa1d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:51.995254 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.995073 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:51.995254 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:51.995107 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-559kf\" (UniqueName: \"kubernetes.io/projected/91a59811-a055-4e42-bc2f-beef27d8fa1d-kube-api-access-559kf\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.095699 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.095599 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91a59811-a055-4e42-bc2f-beef27d8fa1d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.095899 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.095783 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.095899 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.095845 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-559kf\" (UniqueName: \"kubernetes.io/projected/91a59811-a055-4e42-bc2f-beef27d8fa1d-kube-api-access-559kf\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.095899 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.095887 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.096073 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.095910 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.096073 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.095946 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.096073 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.096009 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.096073 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.096065 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91a59811-a055-4e42-bc2f-beef27d8fa1d-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.096277 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.096109 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91a59811-a055-4e42-bc2f-beef27d8fa1d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.096277 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.096212 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.096277 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.096258 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.096390 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.096344 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.096426 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.096393 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.096601 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.096583 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91a59811-a055-4e42-bc2f-beef27d8fa1d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.098412 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.098391 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91a59811-a055-4e42-bc2f-beef27d8fa1d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.098708 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.098693 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91a59811-a055-4e42-bc2f-beef27d8fa1d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.104269 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.104238 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91a59811-a055-4e42-bc2f-beef27d8fa1d-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.104421 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.104404 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-559kf\" (UniqueName: \"kubernetes.io/projected/91a59811-a055-4e42-bc2f-beef27d8fa1d-kube-api-access-559kf\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-lxfhc\" (UID: \"91a59811-a055-4e42-bc2f-beef27d8fa1d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.137428 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.137393 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:52.280933 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.280691 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc"] Jun 02 20:05:52.283564 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:05:52.283532 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a59811_a055_4e42_bc2f_beef27d8fa1d.slice/crio-95a8514f228b79553562abc4ef0f3a2671c0b5d0de0bb91e1749ef5966c3d28e WatchSource:0}: Error finding container 95a8514f228b79553562abc4ef0f3a2671c0b5d0de0bb91e1749ef5966c3d28e: Status 404 returned error can't find the container with id 95a8514f228b79553562abc4ef0f3a2671c0b5d0de0bb91e1749ef5966c3d28e Jun 02 20:05:52.286096 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.286058 2548 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 02 20:05:52.286202 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.286139 2548 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 02 20:05:52.286270 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.286205 2548 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 02 20:05:52.527400 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.527366 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" event={"ID":"91a59811-a055-4e42-bc2f-beef27d8fa1d","Type":"ContainerStarted","Data":"8d1b312c7895a35fb5c08378384c6c1e1a4665cbf3ddd7bf9612c4ccadae4377"} Jun 02 20:05:52.527400 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.527404 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" event={"ID":"91a59811-a055-4e42-bc2f-beef27d8fa1d","Type":"ContainerStarted","Data":"95a8514f228b79553562abc4ef0f3a2671c0b5d0de0bb91e1749ef5966c3d28e"} Jun 02 20:05:52.549223 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:52.549164 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" podStartSLOduration=1.5491487309999998 podStartE2EDuration="1.549148731s" podCreationTimestamp="2026-06-02 20:05:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 20:05:52.548173111 +0000 UTC m=+639.865386366" watchObservedRunningTime="2026-06-02 20:05:52.549148731 +0000 UTC m=+639.866361982" Jun 02 20:05:53.137831 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:53.137780 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:53.142845 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:53.142814 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:53.531702 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:53.531673 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:05:53.532656 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:05:53.532635 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-lxfhc" Jun 02 20:06:08.831428 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:08.831389 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:06:08.834376 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:08.834353 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" Jun 02 20:06:08.836906 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:08.836881 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 02 20:06:08.837037 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:08.837016 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jfdts\"" Jun 02 20:06:08.844086 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:08.844059 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:06:08.878613 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:08.878579 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:06:08.960390 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:08.960349 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ab64f105-21b2-4429-8f89-7c75e7cc8b48-config-file\") pod \"limitador-limitador-78c99df468-zhwbm\" (UID: \"ab64f105-21b2-4429-8f89-7c75e7cc8b48\") " pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" Jun 02 20:06:08.960571 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:08.960416 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbs57\" (UniqueName: \"kubernetes.io/projected/ab64f105-21b2-4429-8f89-7c75e7cc8b48-kube-api-access-bbs57\") pod \"limitador-limitador-78c99df468-zhwbm\" (UID: \"ab64f105-21b2-4429-8f89-7c75e7cc8b48\") " pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" Jun 02 20:06:09.061849 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:09.061801 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bbs57\" (UniqueName: \"kubernetes.io/projected/ab64f105-21b2-4429-8f89-7c75e7cc8b48-kube-api-access-bbs57\") pod \"limitador-limitador-78c99df468-zhwbm\" (UID: \"ab64f105-21b2-4429-8f89-7c75e7cc8b48\") " pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" Jun 02 20:06:09.062055 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:09.061939 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ab64f105-21b2-4429-8f89-7c75e7cc8b48-config-file\") pod \"limitador-limitador-78c99df468-zhwbm\" (UID: \"ab64f105-21b2-4429-8f89-7c75e7cc8b48\") " pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" Jun 02 20:06:09.062563 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:09.062543 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ab64f105-21b2-4429-8f89-7c75e7cc8b48-config-file\") pod \"limitador-limitador-78c99df468-zhwbm\" (UID: \"ab64f105-21b2-4429-8f89-7c75e7cc8b48\") " pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" Jun 02 20:06:09.077397 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:09.077365 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbs57\" (UniqueName: \"kubernetes.io/projected/ab64f105-21b2-4429-8f89-7c75e7cc8b48-kube-api-access-bbs57\") pod \"limitador-limitador-78c99df468-zhwbm\" (UID: \"ab64f105-21b2-4429-8f89-7c75e7cc8b48\") " pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" Jun 02 20:06:09.146613 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:09.146578 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" Jun 02 20:06:09.337428 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:09.337229 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:06:09.340735 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:06:09.340698 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab64f105_21b2_4429_8f89_7c75e7cc8b48.slice/crio-937156be9f1cd3fd3c18b920383ae225bd4f2f9eabb1851b2c49d3d1e8e605a8 WatchSource:0}: Error finding container 937156be9f1cd3fd3c18b920383ae225bd4f2f9eabb1851b2c49d3d1e8e605a8: Status 404 returned error can't find the container with id 937156be9f1cd3fd3c18b920383ae225bd4f2f9eabb1851b2c49d3d1e8e605a8 Jun 02 20:06:09.591213 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:09.591114 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" event={"ID":"ab64f105-21b2-4429-8f89-7c75e7cc8b48","Type":"ContainerStarted","Data":"937156be9f1cd3fd3c18b920383ae225bd4f2f9eabb1851b2c49d3d1e8e605a8"} Jun 02 20:06:12.605342 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:12.605304 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" event={"ID":"ab64f105-21b2-4429-8f89-7c75e7cc8b48","Type":"ContainerStarted","Data":"f33637ba91b9e2d3ca3e95315fe5492b7deacb701b82f0cda79a054652a86d12"} Jun 02 20:06:12.605866 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:12.605365 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" Jun 02 20:06:12.626340 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:12.626288 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" podStartSLOduration=2.154452623 podStartE2EDuration="4.626271936s" podCreationTimestamp="2026-06-02 20:06:08 +0000 UTC" firstStartedPulling="2026-06-02 20:06:09.342850225 +0000 UTC m=+656.660063456" lastFinishedPulling="2026-06-02 20:06:11.81466952 +0000 UTC m=+659.131882769" observedRunningTime="2026-06-02 20:06:12.623975242 +0000 UTC m=+659.941188497" watchObservedRunningTime="2026-06-02 20:06:12.626271936 +0000 UTC m=+659.943485213" Jun 02 20:06:23.610114 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:06:23.610083 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-zhwbm" Jun 02 20:08:14.884612 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:08:14.884573 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:09:33.110228 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:09:33.110186 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:09:55.754471 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:09:55.754430 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:10:09.294344 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.294260 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4"] Jun 02 20:10:09.299105 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.299078 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.302783 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.302733 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 02 20:10:09.303161 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.302797 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7dkwt\"" Jun 02 20:10:09.303315 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.302803 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 02 20:10:09.303478 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.302829 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 02 20:10:09.312454 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.312417 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4"] Jun 02 20:10:09.443067 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.443026 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.443286 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.443089 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cswt2\" (UniqueName: \"kubernetes.io/projected/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-kube-api-access-cswt2\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.443361 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.443331 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.443498 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.443475 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.443613 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.443591 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.443692 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.443639 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.545103 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.545007 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.545103 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.545058 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.545309 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.545157 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.545309 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.545203 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.545309 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.545240 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cswt2\" (UniqueName: \"kubernetes.io/projected/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-kube-api-access-cswt2\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.545309 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.545282 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.545554 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.545520 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.545636 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.545575 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.545636 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.545593 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.547603 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.547570 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.547811 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.547792 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.555404 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.555376 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cswt2\" (UniqueName: \"kubernetes.io/projected/22d40191-4dc1-4be5-a3a4-ec6b2a02edce-kube-api-access-cswt2\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4\" (UID: \"22d40191-4dc1-4be5-a3a4-ec6b2a02edce\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.634578 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.634536 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:09.775375 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.775346 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4"] Jun 02 20:10:09.778086 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:10:09.778059 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d40191_4dc1_4be5_a3a4_ec6b2a02edce.slice/crio-359752ce09de99b6c785a407b184b529abcddadc9f6ca76b9b6584a1aa85678e WatchSource:0}: Error finding container 359752ce09de99b6c785a407b184b529abcddadc9f6ca76b9b6584a1aa85678e: Status 404 returned error can't find the container with id 359752ce09de99b6c785a407b184b529abcddadc9f6ca76b9b6584a1aa85678e Jun 02 20:10:09.779767 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.779732 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 20:10:09.818510 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:09.818424 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:10:10.509941 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:10.509897 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" event={"ID":"22d40191-4dc1-4be5-a3a4-ec6b2a02edce","Type":"ContainerStarted","Data":"359752ce09de99b6c785a407b184b529abcddadc9f6ca76b9b6584a1aa85678e"} Jun 02 20:10:13.988969 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:13.988931 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:10:13.990586 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:13.990562 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:10:14.531245 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:14.531198 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" event={"ID":"22d40191-4dc1-4be5-a3a4-ec6b2a02edce","Type":"ContainerStarted","Data":"ae74be9e0cd000fadec98a76575f02fb8ad382d083cb0abf6f0f288523f98e61"} Jun 02 20:10:15.537497 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:15.537460 2548 generic.go:358] "Generic (PLEG): container finished" podID="22d40191-4dc1-4be5-a3a4-ec6b2a02edce" containerID="ae74be9e0cd000fadec98a76575f02fb8ad382d083cb0abf6f0f288523f98e61" exitCode=0 Jun 02 20:10:15.537960 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:15.537546 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" event={"ID":"22d40191-4dc1-4be5-a3a4-ec6b2a02edce","Type":"ContainerDied","Data":"ae74be9e0cd000fadec98a76575f02fb8ad382d083cb0abf6f0f288523f98e61"} Jun 02 20:10:17.550987 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:17.550949 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" event={"ID":"22d40191-4dc1-4be5-a3a4-ec6b2a02edce","Type":"ContainerStarted","Data":"227f153ebb6590ba93d6919bcf376fd8c471e2c4c15075ec3e9f76100a9f7a29"} Jun 02 20:10:17.573452 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:17.573396 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" podStartSLOduration=1.505847453 podStartE2EDuration="8.573379758s" podCreationTimestamp="2026-06-02 20:10:09 +0000 UTC" firstStartedPulling="2026-06-02 20:10:09.779881902 +0000 UTC m=+897.097095147" lastFinishedPulling="2026-06-02 20:10:16.847414216 +0000 UTC m=+904.164627452" observedRunningTime="2026-06-02 20:10:17.572993092 +0000 UTC m=+904.890206346" watchObservedRunningTime="2026-06-02 20:10:17.573379758 +0000 UTC m=+904.890593013" Jun 02 20:10:19.635172 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:19.635129 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:19.635172 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:19.635171 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:19.648064 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:19.648028 2548 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:20.576039 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:20.576006 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4" Jun 02 20:10:21.409577 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:21.409536 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:10:41.035022 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:41.034981 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:10:48.313753 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:10:48.313699 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:12:15.412214 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:12:15.412175 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:12:26.308238 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:12:26.308202 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:12:35.039514 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:12:35.039479 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:12:45.614416 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:12:45.614374 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:12:54.215633 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:12:54.215599 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:13:04.613800 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:13:04.613714 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:14:18.941177 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:14:18.941138 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:14:34.807142 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:14:34.807053 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:15:00.144521 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:00.144481 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29673855-z7vvx"] Jun 02 20:15:00.147065 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:00.147046 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29673855-z7vvx" Jun 02 20:15:00.149650 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:00.149622 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-hdfmm\"" Jun 02 20:15:00.166712 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:00.166677 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29673855-z7vvx"] Jun 02 20:15:00.271326 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:00.271275 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47v26\" (UniqueName: \"kubernetes.io/projected/daa68612-1c69-451f-a4e5-3f2213437d3e-kube-api-access-47v26\") pod \"maas-api-key-cleanup-29673855-z7vvx\" (UID: \"daa68612-1c69-451f-a4e5-3f2213437d3e\") " pod="opendatahub/maas-api-key-cleanup-29673855-z7vvx" Jun 02 20:15:00.372249 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:00.372202 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-47v26\" (UniqueName: \"kubernetes.io/projected/daa68612-1c69-451f-a4e5-3f2213437d3e-kube-api-access-47v26\") pod \"maas-api-key-cleanup-29673855-z7vvx\" (UID: \"daa68612-1c69-451f-a4e5-3f2213437d3e\") " pod="opendatahub/maas-api-key-cleanup-29673855-z7vvx" Jun 02 20:15:00.381303 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:00.381268 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47v26\" (UniqueName: \"kubernetes.io/projected/daa68612-1c69-451f-a4e5-3f2213437d3e-kube-api-access-47v26\") pod \"maas-api-key-cleanup-29673855-z7vvx\" (UID: \"daa68612-1c69-451f-a4e5-3f2213437d3e\") " pod="opendatahub/maas-api-key-cleanup-29673855-z7vvx" Jun 02 20:15:00.457514 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:00.457472 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29673855-z7vvx" Jun 02 20:15:00.596579 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:00.596551 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29673855-z7vvx"] Jun 02 20:15:00.599194 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:15:00.599163 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa68612_1c69_451f_a4e5_3f2213437d3e.slice/crio-19e44e5119798725ce3a935d94563c36e9c0fa9dd8de3162c157407033783c25 WatchSource:0}: Error finding container 19e44e5119798725ce3a935d94563c36e9c0fa9dd8de3162c157407033783c25: Status 404 returned error can't find the container with id 19e44e5119798725ce3a935d94563c36e9c0fa9dd8de3162c157407033783c25 Jun 02 20:15:00.668634 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:00.668598 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29673855-z7vvx" event={"ID":"daa68612-1c69-451f-a4e5-3f2213437d3e","Type":"ContainerStarted","Data":"19e44e5119798725ce3a935d94563c36e9c0fa9dd8de3162c157407033783c25"} Jun 02 20:15:03.682461 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:03.682422 2548 generic.go:358] "Generic (PLEG): container finished" podID="daa68612-1c69-451f-a4e5-3f2213437d3e" containerID="da06d2c15b374d28fdf8467f5b94d8d1b7cb5ccc9757d8da91e0f5829c5b6d85" exitCode=0 Jun 02 20:15:03.682873 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:03.682513 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29673855-z7vvx" event={"ID":"daa68612-1c69-451f-a4e5-3f2213437d3e","Type":"ContainerDied","Data":"da06d2c15b374d28fdf8467f5b94d8d1b7cb5ccc9757d8da91e0f5829c5b6d85"} Jun 02 20:15:04.821328 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:04.821303 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29673855-z7vvx" Jun 02 20:15:04.919098 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:04.919055 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47v26\" (UniqueName: \"kubernetes.io/projected/daa68612-1c69-451f-a4e5-3f2213437d3e-kube-api-access-47v26\") pod \"daa68612-1c69-451f-a4e5-3f2213437d3e\" (UID: \"daa68612-1c69-451f-a4e5-3f2213437d3e\") " Jun 02 20:15:04.921333 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:04.921301 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa68612-1c69-451f-a4e5-3f2213437d3e-kube-api-access-47v26" (OuterVolumeSpecName: "kube-api-access-47v26") pod "daa68612-1c69-451f-a4e5-3f2213437d3e" (UID: "daa68612-1c69-451f-a4e5-3f2213437d3e"). InnerVolumeSpecName "kube-api-access-47v26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:15:05.020150 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:05.020060 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47v26\" (UniqueName: \"kubernetes.io/projected/daa68612-1c69-451f-a4e5-3f2213437d3e-kube-api-access-47v26\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 20:15:05.691968 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:05.691929 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29673855-z7vvx" event={"ID":"daa68612-1c69-451f-a4e5-3f2213437d3e","Type":"ContainerDied","Data":"19e44e5119798725ce3a935d94563c36e9c0fa9dd8de3162c157407033783c25"} Jun 02 20:15:05.691968 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:05.691955 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29673855-z7vvx" Jun 02 20:15:05.692268 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:05.691962 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19e44e5119798725ce3a935d94563c36e9c0fa9dd8de3162c157407033783c25" Jun 02 20:15:14.023939 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:14.023906 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:15:14.024997 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:14.024975 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:15:14.907091 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:14.907047 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:15:31.205663 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:31.205623 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:15:45.303955 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:15:45.303916 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:16:02.912560 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:16:02.912479 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:16:30.508408 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:16:30.508369 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:16:36.507525 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:16:36.507478 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:17:12.625860 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:17:12.625819 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:17:21.921142 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:17:21.921102 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:17:38.505899 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:17:38.505810 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:17:47.507454 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:17:47.507413 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:18:04.315021 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:18:04.314978 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:18:10.705068 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:18:10.705030 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:18:45.436541 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:18:45.436498 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:18:54.036435 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:18:54.036392 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:19:02.305279 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:19:02.305188 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:19:09.914781 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:19:09.914720 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:19:19.108221 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:19:19.108182 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:19:35.611022 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:19:35.610983 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:19:50.104060 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:19:50.104019 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:20:14.056649 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:20:14.056616 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:20:14.063071 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:20:14.063037 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:20:38.820337 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:20:38.820239 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:20:46.904768 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:20:46.904693 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:20:52.222504 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:20:52.222465 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:21:00.923846 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:21:00.923803 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:21:09.511024 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:21:09.510985 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:21:18.837592 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:21:18.837554 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:21:27.623214 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:21:27.623179 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:21:36.409973 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:21:36.409930 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:21:45.225615 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:21:45.225574 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:21:54.313428 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:21:54.313396 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:22:02.819993 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:22:02.819908 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:22:12.028269 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:22:12.028233 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:22:21.113441 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:22:21.113397 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:22:30.925851 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:22:30.925808 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:22:41.437270 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:22:41.437234 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:22:47.286405 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:22:47.286357 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:22:55.727858 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:22:55.727817 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:23:04.716155 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:23:04.716114 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:25:14.092096 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:25:14.092070 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:25:14.095593 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:25:14.095567 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:25:24.013870 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:25:24.013825 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:25:30.610637 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:25:30.610596 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:25:57.920016 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:25:57.919979 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:26:07.918593 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:26:07.918546 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:26:17.358816 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:26:17.358772 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:26:26.218140 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:26:26.218095 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:26:34.910863 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:26:34.910768 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:26:45.505608 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:26:45.505573 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:26:54.139264 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:26:54.139222 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:27:05.309451 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:27:05.309413 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:27:14.225600 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:27:14.225560 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:27:25.318849 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:27:25.318818 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:27:33.917603 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:27:33.917562 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:28:10.009437 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:28:10.009340 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:28:51.327237 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:28:51.327197 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:29:00.319222 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:29:00.319181 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:29:08.818471 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:29:08.818429 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:29:18.117785 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:29:18.117732 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:29:27.112755 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:29:27.112701 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:29:40.415088 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:29:40.414995 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:29:49.008368 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:29:49.008326 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:29:56.824486 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:29:56.824442 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:30:00.143291 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.143246 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29673870-xxrfq"] Jun 02 20:30:00.143845 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.143688 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daa68612-1c69-451f-a4e5-3f2213437d3e" containerName="cleanup" Jun 02 20:30:00.143845 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.143699 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="daa68612-1c69-451f-a4e5-3f2213437d3e" containerName="cleanup" Jun 02 20:30:00.143845 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.143803 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="daa68612-1c69-451f-a4e5-3f2213437d3e" containerName="cleanup" Jun 02 20:30:00.147079 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.147060 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29673870-xxrfq" Jun 02 20:30:00.149967 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.149938 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-hdfmm\"" Jun 02 20:30:00.167326 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.167295 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29673870-xxrfq"] Jun 02 20:30:00.240728 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.240686 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gggzz\" (UniqueName: \"kubernetes.io/projected/47417fd8-6b6b-4957-9044-2a00255ad879-kube-api-access-gggzz\") pod \"maas-api-key-cleanup-29673870-xxrfq\" (UID: \"47417fd8-6b6b-4957-9044-2a00255ad879\") " pod="opendatahub/maas-api-key-cleanup-29673870-xxrfq" Jun 02 20:30:00.342291 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.342246 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gggzz\" (UniqueName: \"kubernetes.io/projected/47417fd8-6b6b-4957-9044-2a00255ad879-kube-api-access-gggzz\") pod \"maas-api-key-cleanup-29673870-xxrfq\" (UID: \"47417fd8-6b6b-4957-9044-2a00255ad879\") " pod="opendatahub/maas-api-key-cleanup-29673870-xxrfq" Jun 02 20:30:00.352659 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.352617 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gggzz\" (UniqueName: \"kubernetes.io/projected/47417fd8-6b6b-4957-9044-2a00255ad879-kube-api-access-gggzz\") pod \"maas-api-key-cleanup-29673870-xxrfq\" (UID: \"47417fd8-6b6b-4957-9044-2a00255ad879\") " pod="opendatahub/maas-api-key-cleanup-29673870-xxrfq" Jun 02 20:30:00.457613 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.457578 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29673870-xxrfq" Jun 02 20:30:00.604249 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.604221 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29673870-xxrfq"] Jun 02 20:30:00.606884 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:30:00.606853 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47417fd8_6b6b_4957_9044_2a00255ad879.slice/crio-bc72659aa13b8b511feb718c8006513a14c558fb3514cf51fdbb347d3d572c52 WatchSource:0}: Error finding container bc72659aa13b8b511feb718c8006513a14c558fb3514cf51fdbb347d3d572c52: Status 404 returned error can't find the container with id bc72659aa13b8b511feb718c8006513a14c558fb3514cf51fdbb347d3d572c52 Jun 02 20:30:00.608913 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:00.608893 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 20:30:01.179398 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:01.179363 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29673870-xxrfq" event={"ID":"47417fd8-6b6b-4957-9044-2a00255ad879","Type":"ContainerStarted","Data":"0d59e9ec67e70a0b7672d5d08de05ad54e6594661a5faaf454d6e69a98b9a058"} Jun 02 20:30:01.179398 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:01.179399 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29673870-xxrfq" event={"ID":"47417fd8-6b6b-4957-9044-2a00255ad879","Type":"ContainerStarted","Data":"bc72659aa13b8b511feb718c8006513a14c558fb3514cf51fdbb347d3d572c52"} Jun 02 20:30:01.207823 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:01.207773 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29673870-xxrfq" podStartSLOduration=1.207755794 podStartE2EDuration="1.207755794s" podCreationTimestamp="2026-06-02 20:30:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 20:30:01.20559491 +0000 UTC m=+2088.522808167" watchObservedRunningTime="2026-06-02 20:30:01.207755794 +0000 UTC m=+2088.524969043" Jun 02 20:30:02.184776 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:02.184720 2548 generic.go:358] "Generic (PLEG): container finished" podID="47417fd8-6b6b-4957-9044-2a00255ad879" containerID="0d59e9ec67e70a0b7672d5d08de05ad54e6594661a5faaf454d6e69a98b9a058" exitCode=0 Jun 02 20:30:02.185186 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:02.184807 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29673870-xxrfq" event={"ID":"47417fd8-6b6b-4957-9044-2a00255ad879","Type":"ContainerDied","Data":"0d59e9ec67e70a0b7672d5d08de05ad54e6594661a5faaf454d6e69a98b9a058"} Jun 02 20:30:03.330046 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:03.330020 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29673870-xxrfq" Jun 02 20:30:03.374217 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:03.374178 2548 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gggzz\" (UniqueName: \"kubernetes.io/projected/47417fd8-6b6b-4957-9044-2a00255ad879-kube-api-access-gggzz\") pod \"47417fd8-6b6b-4957-9044-2a00255ad879\" (UID: \"47417fd8-6b6b-4957-9044-2a00255ad879\") " Jun 02 20:30:03.376531 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:03.376491 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47417fd8-6b6b-4957-9044-2a00255ad879-kube-api-access-gggzz" (OuterVolumeSpecName: "kube-api-access-gggzz") pod "47417fd8-6b6b-4957-9044-2a00255ad879" (UID: "47417fd8-6b6b-4957-9044-2a00255ad879"). InnerVolumeSpecName "kube-api-access-gggzz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:30:03.475715 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:03.475614 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gggzz\" (UniqueName: \"kubernetes.io/projected/47417fd8-6b6b-4957-9044-2a00255ad879-kube-api-access-gggzz\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jun 02 20:30:04.193929 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:04.193901 2548 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29673870-xxrfq" Jun 02 20:30:04.193929 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:04.193913 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29673870-xxrfq" event={"ID":"47417fd8-6b6b-4957-9044-2a00255ad879","Type":"ContainerDied","Data":"bc72659aa13b8b511feb718c8006513a14c558fb3514cf51fdbb347d3d572c52"} Jun 02 20:30:04.194202 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:04.193955 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc72659aa13b8b511feb718c8006513a14c558fb3514cf51fdbb347d3d572c52" Jun 02 20:30:07.908109 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:07.908073 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:30:14.122527 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:14.122497 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:30:14.127078 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:14.127056 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:30:14.615878 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:14.615836 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:30:25.217892 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:25.217853 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:30:36.606677 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:36.606636 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:30:55.810924 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:30:55.810874 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:31:01.817299 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:31:01.817210 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:31:12.815955 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:31:12.815921 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:31:19.213279 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:31:19.213237 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:31:36.245423 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:31:36.245388 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:31:45.714400 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:31:45.714357 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:31:54.735285 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:31:54.735244 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:32:01.627028 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:32:01.626991 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:32:12.510560 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:32:12.510525 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:32:19.312789 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:32:19.312736 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:32:30.207840 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:32:30.204938 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:32:43.012106 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:32:43.012063 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:32:50.839537 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:32:50.839501 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:33:04.808117 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:33:04.808080 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:33:14.914106 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:33:14.914057 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:33:25.534070 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:33:25.534025 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:33:34.004444 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:33:34.004401 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:33:41.811437 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:33:41.811390 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:34:00.507340 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:34:00.507250 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:34:09.609039 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:34:09.608994 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:34:18.308294 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:34:18.308260 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:34:28.613238 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:34:28.613198 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:34:51.098114 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:34:51.098073 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:35:03.805029 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:03.804986 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-zhwbm"] Jun 02 20:35:14.152757 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:14.152717 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:35:14.158529 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:14.158508 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:35:14.533102 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:14.532995 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-54b5c7c656-qfgpk_31ec5341-8c1b-437c-aa4e-130e89789768/nginx/0.log" Jun 02 20:35:15.029624 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:15.029592 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29673855-z7vvx_daa68612-1c69-451f-a4e5-3f2213437d3e/cleanup/0.log" Jun 02 20:35:15.143861 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:15.143827 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29673870-xxrfq_47417fd8-6b6b-4957-9044-2a00255ad879/cleanup/0.log" Jun 02 20:35:15.758147 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:15.758114 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6977cb796-js76j_f43fcdad-b0ed-4457-879f-62b470c72622/copy-manifests/0.log" Jun 02 20:35:15.765965 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:15.765939 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6977cb796-js76j_f43fcdad-b0ed-4457-879f-62b470c72622/manager/0.log" Jun 02 20:35:17.498878 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:17.498840 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-m9r26_5d0a7299-4f6c-4900-8245-23223f2cbc12/manager/0.log" Jun 02 20:35:17.615419 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:17.615388 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-tzrtf_c27f5c97-f179-4ad9-acbb-24a12b45fa13/manager/0.log" Jun 02 20:35:17.864028 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:17.863937 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5x7xq_a4b29350-a492-4163-8f50-7521395ac15e/registry-server/0.log" Jun 02 20:35:18.119309 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:18.119223 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-zhwbm_ab64f105-21b2-4429-8f89-7c75e7cc8b48/limitador/0.log" Jun 02 20:35:18.241767 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:18.241723 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-nrpkb_8d93ac9c-93d5-4d6b-ba6a-2017ce6f4a8d/manager/0.log" Jun 02 20:35:18.602416 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:18.602382 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf_00a7259e-d63c-4b89-a3a7-dfc037790e60/istio-proxy/0.log" Jun 02 20:35:19.059611 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:19.059580 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-lxfhc_91a59811-a055-4e42-bc2f-beef27d8fa1d/istio-proxy/0.log" Jun 02 20:35:19.885107 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:19.885058 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4_22d40191-4dc1-4be5-a3a4-ec6b2a02edce/storage-initializer/0.log" Jun 02 20:35:19.892378 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:19.892351 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-zfss4_22d40191-4dc1-4be5-a3a4-ec6b2a02edce/main/0.log" Jun 02 20:35:24.831249 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.831211 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jnp2s/must-gather-qvvqs"] Jun 02 20:35:24.831822 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.831800 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47417fd8-6b6b-4957-9044-2a00255ad879" containerName="cleanup" Jun 02 20:35:24.831959 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.831861 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="47417fd8-6b6b-4957-9044-2a00255ad879" containerName="cleanup" Jun 02 20:35:24.832067 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.831988 2548 memory_manager.go:356] "RemoveStaleState removing state" podUID="47417fd8-6b6b-4957-9044-2a00255ad879" containerName="cleanup" Jun 02 20:35:24.835417 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.835394 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jnp2s/must-gather-qvvqs" Jun 02 20:35:24.842301 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.842006 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jnp2s\"/\"openshift-service-ca.crt\"" Jun 02 20:35:24.842460 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.842442 2548 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jnp2s\"/\"kube-root-ca.crt\"" Jun 02 20:35:24.843658 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.843641 2548 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jnp2s\"/\"default-dockercfg-bc8vx\"" Jun 02 20:35:24.886327 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.886291 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8rlx\" (UniqueName: \"kubernetes.io/projected/fb7a5032-a550-4dcc-a087-e697dfba1541-kube-api-access-b8rlx\") pod \"must-gather-qvvqs\" (UID: \"fb7a5032-a550-4dcc-a087-e697dfba1541\") " pod="openshift-must-gather-jnp2s/must-gather-qvvqs" Jun 02 20:35:24.886537 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.886339 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fb7a5032-a550-4dcc-a087-e697dfba1541-must-gather-output\") pod \"must-gather-qvvqs\" (UID: \"fb7a5032-a550-4dcc-a087-e697dfba1541\") " pod="openshift-must-gather-jnp2s/must-gather-qvvqs" Jun 02 20:35:24.921315 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.921283 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jnp2s/must-gather-qvvqs"] Jun 02 20:35:24.987199 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.987161 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b8rlx\" (UniqueName: \"kubernetes.io/projected/fb7a5032-a550-4dcc-a087-e697dfba1541-kube-api-access-b8rlx\") pod \"must-gather-qvvqs\" (UID: \"fb7a5032-a550-4dcc-a087-e697dfba1541\") " pod="openshift-must-gather-jnp2s/must-gather-qvvqs" Jun 02 20:35:24.987380 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.987211 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fb7a5032-a550-4dcc-a087-e697dfba1541-must-gather-output\") pod \"must-gather-qvvqs\" (UID: \"fb7a5032-a550-4dcc-a087-e697dfba1541\") " pod="openshift-must-gather-jnp2s/must-gather-qvvqs" Jun 02 20:35:24.987566 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:24.987546 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fb7a5032-a550-4dcc-a087-e697dfba1541-must-gather-output\") pod \"must-gather-qvvqs\" (UID: \"fb7a5032-a550-4dcc-a087-e697dfba1541\") " pod="openshift-must-gather-jnp2s/must-gather-qvvqs" Jun 02 20:35:25.009341 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:25.009311 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8rlx\" (UniqueName: \"kubernetes.io/projected/fb7a5032-a550-4dcc-a087-e697dfba1541-kube-api-access-b8rlx\") pod \"must-gather-qvvqs\" (UID: \"fb7a5032-a550-4dcc-a087-e697dfba1541\") " pod="openshift-must-gather-jnp2s/must-gather-qvvqs" Jun 02 20:35:25.144966 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:25.144871 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jnp2s/must-gather-qvvqs" Jun 02 20:35:25.290433 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:25.290409 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jnp2s/must-gather-qvvqs"] Jun 02 20:35:25.292790 ip-10-0-129-157 kubenswrapper[2548]: W0602 20:35:25.292755 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb7a5032_a550_4dcc_a087_e697dfba1541.slice/crio-886baaf10fe86a182c63c4a1268d20f90cabd6c34ecefe50be3beb7919553ccf WatchSource:0}: Error finding container 886baaf10fe86a182c63c4a1268d20f90cabd6c34ecefe50be3beb7919553ccf: Status 404 returned error can't find the container with id 886baaf10fe86a182c63c4a1268d20f90cabd6c34ecefe50be3beb7919553ccf Jun 02 20:35:25.294647 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:25.294615 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 20:35:25.481321 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:25.481283 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jnp2s/must-gather-qvvqs" event={"ID":"fb7a5032-a550-4dcc-a087-e697dfba1541","Type":"ContainerStarted","Data":"886baaf10fe86a182c63c4a1268d20f90cabd6c34ecefe50be3beb7919553ccf"} Jun 02 20:35:26.486826 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:26.486788 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jnp2s/must-gather-qvvqs" event={"ID":"fb7a5032-a550-4dcc-a087-e697dfba1541","Type":"ContainerStarted","Data":"515e1f9a66f40fe2b9691ae548879f9da1a504bb5b5d294372c606829760c7f6"} Jun 02 20:35:26.486826 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:26.486826 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jnp2s/must-gather-qvvqs" event={"ID":"fb7a5032-a550-4dcc-a087-e697dfba1541","Type":"ContainerStarted","Data":"5c880412363c5bdf48246cf6a4024d8d07ac0f3068c49d16aad56b8e79fc43a8"} Jun 02 20:35:26.511707 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:26.511641 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-jnp2s/must-gather-qvvqs" podStartSLOduration=1.7706272589999998 podStartE2EDuration="2.511622803s" podCreationTimestamp="2026-06-02 20:35:24 +0000 UTC" firstStartedPulling="2026-06-02 20:35:25.294850175 +0000 UTC m=+2412.612063425" lastFinishedPulling="2026-06-02 20:35:26.035845733 +0000 UTC m=+2413.353058969" observedRunningTime="2026-06-02 20:35:26.509811809 +0000 UTC m=+2413.827025062" watchObservedRunningTime="2026-06-02 20:35:26.511622803 +0000 UTC m=+2413.828836058" Jun 02 20:35:28.117090 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:28.117054 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xzfr6_39338125-49b0-4a5f-9591-ca49089d0da5/global-pull-secret-syncer/0.log" Jun 02 20:35:28.211247 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:28.211218 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2ng7z_286e8706-b9e4-4ae9-b67f-3ddbcf5877a6/konnectivity-agent/0.log" Jun 02 20:35:28.263795 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:28.263759 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-157.ec2.internal_2dcf4b3d1e27492b4997b358cd9a9f92/haproxy/0.log" Jun 02 20:35:33.448448 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:33.448411 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-m9r26_5d0a7299-4f6c-4900-8245-23223f2cbc12/manager/0.log" Jun 02 20:35:33.473811 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:33.473779 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-tzrtf_c27f5c97-f179-4ad9-acbb-24a12b45fa13/manager/0.log" Jun 02 20:35:33.552962 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:33.552927 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5x7xq_a4b29350-a492-4163-8f50-7521395ac15e/registry-server/0.log" Jun 02 20:35:33.632999 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:33.632969 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-zhwbm_ab64f105-21b2-4429-8f89-7c75e7cc8b48/limitador/0.log" Jun 02 20:35:33.723698 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:33.723567 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-nrpkb_8d93ac9c-93d5-4d6b-ba6a-2017ce6f4a8d/manager/0.log" Jun 02 20:35:35.273198 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.273097 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfe250e1-be31-41b6-8071-0b2d383f198b/alertmanager/0.log" Jun 02 20:35:35.298849 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.298784 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfe250e1-be31-41b6-8071-0b2d383f198b/config-reloader/0.log" Jun 02 20:35:35.321885 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.321800 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfe250e1-be31-41b6-8071-0b2d383f198b/kube-rbac-proxy-web/0.log" Jun 02 20:35:35.348982 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.348954 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfe250e1-be31-41b6-8071-0b2d383f198b/kube-rbac-proxy/0.log" Jun 02 20:35:35.372422 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.372390 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfe250e1-be31-41b6-8071-0b2d383f198b/kube-rbac-proxy-metric/0.log" Jun 02 20:35:35.397519 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.397492 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfe250e1-be31-41b6-8071-0b2d383f198b/prom-label-proxy/0.log" Jun 02 20:35:35.421769 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.421724 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfe250e1-be31-41b6-8071-0b2d383f198b/init-config-reloader/0.log" Jun 02 20:35:35.469002 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.468974 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-69b6cbdb88-z57gd_cc66e3d3-9282-4836-af26-6438bb0179dc/cluster-monitoring-operator/0.log" Jun 02 20:35:35.502335 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.502298 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-g8c8m_62c1ac6c-eef9-4d69-8835-b25a23d64e0c/kube-state-metrics/0.log" Jun 02 20:35:35.522965 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.522930 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-g8c8m_62c1ac6c-eef9-4d69-8835-b25a23d64e0c/kube-rbac-proxy-main/0.log" Jun 02 20:35:35.547917 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.547819 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-g8c8m_62c1ac6c-eef9-4d69-8835-b25a23d64e0c/kube-rbac-proxy-self/0.log" Jun 02 20:35:35.579075 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.579038 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-548f54f5f7-hqrg8_32785a32-52d3-4401-ab3c-915abb3835e7/metrics-server/0.log" Jun 02 20:35:35.606167 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.606142 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-7f697_6cfc3c1d-ea45-4b98-a84b-1ee69bf9dc39/monitoring-plugin/0.log" Jun 02 20:35:35.787436 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.787397 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lqhbt_f5800e54-11e6-4369-ab41-1d8454b4caee/node-exporter/0.log" Jun 02 20:35:35.809527 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.809438 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lqhbt_f5800e54-11e6-4369-ab41-1d8454b4caee/kube-rbac-proxy/0.log" Jun 02 20:35:35.832135 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.832108 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lqhbt_f5800e54-11e6-4369-ab41-1d8454b4caee/init-textfile/0.log" Jun 02 20:35:35.859326 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.859285 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-r2brz_2115b62d-ce15-4d1b-8fce-7dbee21ace94/kube-rbac-proxy-main/0.log" Jun 02 20:35:35.881182 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.881151 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-r2brz_2115b62d-ce15-4d1b-8fce-7dbee21ace94/kube-rbac-proxy-self/0.log" Jun 02 20:35:35.903721 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.903689 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-r2brz_2115b62d-ce15-4d1b-8fce-7dbee21ace94/openshift-state-metrics/0.log" Jun 02 20:35:35.941818 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.941786 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1aa43051-e1c9-4551-901d-2ae0d32143b5/prometheus/0.log" Jun 02 20:35:35.964733 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.964701 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1aa43051-e1c9-4551-901d-2ae0d32143b5/config-reloader/0.log" Jun 02 20:35:35.997989 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:35.997949 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1aa43051-e1c9-4551-901d-2ae0d32143b5/thanos-sidecar/0.log" Jun 02 20:35:36.040358 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:36.040330 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1aa43051-e1c9-4551-901d-2ae0d32143b5/kube-rbac-proxy-web/0.log" Jun 02 20:35:36.063702 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:36.063623 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1aa43051-e1c9-4551-901d-2ae0d32143b5/kube-rbac-proxy/0.log" Jun 02 20:35:36.088571 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:36.088538 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1aa43051-e1c9-4551-901d-2ae0d32143b5/kube-rbac-proxy-thanos/0.log" Jun 02 20:35:36.111658 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:36.111624 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1aa43051-e1c9-4551-901d-2ae0d32143b5/init-config-reloader/0.log" Jun 02 20:35:36.142725 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:36.142692 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-54g6l_db78ab6d-7ab0-483d-a46f-98d4f03a07a3/prometheus-operator/0.log" Jun 02 20:35:36.171510 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:36.171472 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-54g6l_db78ab6d-7ab0-483d-a46f-98d4f03a07a3/kube-rbac-proxy/0.log" Jun 02 20:35:36.225499 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:36.225471 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74857bcbf9-lhk8x_18f8431c-27c3-4cb4-ab0f-b6c5b7679138/telemeter-client/0.log" Jun 02 20:35:36.246412 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:36.246383 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74857bcbf9-lhk8x_18f8431c-27c3-4cb4-ab0f-b6c5b7679138/reload/0.log" Jun 02 20:35:36.270548 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:36.270519 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74857bcbf9-lhk8x_18f8431c-27c3-4cb4-ab0f-b6c5b7679138/kube-rbac-proxy/0.log" Jun 02 20:35:38.965784 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:38.965732 2548 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79"] Jun 02 20:35:38.973591 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:38.973567 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:38.980434 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:38.980407 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79"] Jun 02 20:35:39.052839 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.052790 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7n6j\" (UniqueName: \"kubernetes.io/projected/ac684e2b-bd71-4690-85f0-932fe5cb885f-kube-api-access-v7n6j\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.053009 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.052866 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac684e2b-bd71-4690-85f0-932fe5cb885f-sys\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.053009 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.052915 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac684e2b-bd71-4690-85f0-932fe5cb885f-proc\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.053103 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.053085 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac684e2b-bd71-4690-85f0-932fe5cb885f-lib-modules\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.053139 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.053117 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac684e2b-bd71-4690-85f0-932fe5cb885f-podres\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.154684 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.154632 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac684e2b-bd71-4690-85f0-932fe5cb885f-lib-modules\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.154684 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.154680 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac684e2b-bd71-4690-85f0-932fe5cb885f-podres\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.154973 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.154725 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v7n6j\" (UniqueName: \"kubernetes.io/projected/ac684e2b-bd71-4690-85f0-932fe5cb885f-kube-api-access-v7n6j\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.154973 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.154765 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac684e2b-bd71-4690-85f0-932fe5cb885f-sys\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.154973 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.154784 2548 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac684e2b-bd71-4690-85f0-932fe5cb885f-proc\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.154973 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.154852 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac684e2b-bd71-4690-85f0-932fe5cb885f-lib-modules\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.154973 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.154871 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac684e2b-bd71-4690-85f0-932fe5cb885f-proc\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.154973 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.154872 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac684e2b-bd71-4690-85f0-932fe5cb885f-podres\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.154973 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.154895 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac684e2b-bd71-4690-85f0-932fe5cb885f-sys\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.164646 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.164606 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7n6j\" (UniqueName: \"kubernetes.io/projected/ac684e2b-bd71-4690-85f0-932fe5cb885f-kube-api-access-v7n6j\") pod \"perf-node-gather-daemonset-k8n79\" (UID: \"ac684e2b-bd71-4690-85f0-932fe5cb885f\") " pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.288440 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.288330 2548 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:39.451303 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.450935 2548 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79"] Jun 02 20:35:39.562170 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:39.562134 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" event={"ID":"ac684e2b-bd71-4690-85f0-932fe5cb885f","Type":"ContainerStarted","Data":"6ae4449f3d02306ba5b869e6a7da1984dfdbccdba43b47932c77abdf9cf64fa1"} Jun 02 20:35:40.082403 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:40.082371 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rmbfk_7952dddc-f51b-4a0c-8a23-feecfec0457a/dns/0.log" Jun 02 20:35:40.104331 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:40.104239 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rmbfk_7952dddc-f51b-4a0c-8a23-feecfec0457a/kube-rbac-proxy/0.log" Jun 02 20:35:40.162709 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:40.162487 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lz2mg_c3674c37-2e21-4799-9545-d69c1985b721/dns-node-resolver/0.log" Jun 02 20:35:40.568800 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:40.568728 2548 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" event={"ID":"ac684e2b-bd71-4690-85f0-932fe5cb885f","Type":"ContainerStarted","Data":"d341791f95722e5899a29b20579f7b8514531d2d138db4ca84e0b4d69a56fcc4"} Jun 02 20:35:40.569020 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:40.568940 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:40.588080 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:40.588030 2548 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" podStartSLOduration=2.58801506 podStartE2EDuration="2.58801506s" podCreationTimestamp="2026-06-02 20:35:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 20:35:40.58510572 +0000 UTC m=+2427.902318975" watchObservedRunningTime="2026-06-02 20:35:40.58801506 +0000 UTC m=+2427.905228317" Jun 02 20:35:40.737988 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:40.737948 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kcv4k_861f1595-8ced-49bc-b4e3-2db727f3bb25/node-ca/0.log" Jun 02 20:35:41.628388 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:41.628360 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bhbkrf_00a7259e-d63c-4b89-a3a7-dfc037790e60/istio-proxy/0.log" Jun 02 20:35:41.837469 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:41.837436 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-lxfhc_91a59811-a055-4e42-bc2f-beef27d8fa1d/istio-proxy/0.log" Jun 02 20:35:42.604432 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:42.604403 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jjfnh_3bade01b-4276-40ba-b81a-d607f88064e2/serve-healthcheck-canary/0.log" Jun 02 20:35:43.070167 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:43.070141 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-788bf58df5-kgq9t_1493c956-abd3-4deb-be9c-9ac0eb2eb886/insights-operator/1.log" Jun 02 20:35:43.070657 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:43.070637 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-788bf58df5-kgq9t_1493c956-abd3-4deb-be9c-9ac0eb2eb886/insights-operator/0.log" Jun 02 20:35:43.245240 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:43.245214 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmzjz_c332bbad-564f-4d40-84c6-4fff506f49eb/kube-rbac-proxy/0.log" Jun 02 20:35:43.266669 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:43.266644 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmzjz_c332bbad-564f-4d40-84c6-4fff506f49eb/exporter/0.log" Jun 02 20:35:43.288893 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:43.288863 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmzjz_c332bbad-564f-4d40-84c6-4fff506f49eb/extractor/0.log" Jun 02 20:35:45.533865 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:45.533837 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-54b5c7c656-qfgpk_31ec5341-8c1b-437c-aa4e-130e89789768/nginx/0.log" Jun 02 20:35:45.708522 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:45.708491 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29673855-z7vvx_daa68612-1c69-451f-a4e5-3f2213437d3e/cleanup/0.log" Jun 02 20:35:45.730974 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:45.730941 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29673870-xxrfq_47417fd8-6b6b-4957-9044-2a00255ad879/cleanup/0.log" Jun 02 20:35:45.979785 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:45.979726 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6977cb796-js76j_f43fcdad-b0ed-4457-879f-62b470c72622/manager/0.log" Jun 02 20:35:46.000172 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:46.000140 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6977cb796-js76j_f43fcdad-b0ed-4457-879f-62b470c72622/copy-manifests/0.log" Jun 02 20:35:46.584228 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:46.584201 2548 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jnp2s/perf-node-gather-daemonset-k8n79" Jun 02 20:35:47.724407 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:47.724378 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-69d845485f-4m69k_60887d2f-199c-4d20-b12d-e36308588b6f/manager/0.log" Jun 02 20:35:47.789814 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:47.789778 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-58rbf_1ae7b561-29d7-4743-9039-858d328baa1a/openshift-lws-operator/0.log" Jun 02 20:35:53.064827 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:53.064781 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-khsh7_76e3be1c-425f-407d-8805-bc27d505f922/kube-storage-version-migrator-operator/1.log" Jun 02 20:35:53.065801 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:53.065775 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-khsh7_76e3be1c-425f-407d-8805-bc27d505f922/kube-storage-version-migrator-operator/0.log" Jun 02 20:35:54.162654 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:54.162608 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-52r6t_372120bf-121b-4dff-b2b0-f89a944a42dc/kube-multus/0.log" Jun 02 20:35:54.392568 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:54.392529 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5w6p_4271c2ac-658b-4949-b178-ac6530158698/kube-multus-additional-cni-plugins/0.log" Jun 02 20:35:54.420984 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:54.420903 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5w6p_4271c2ac-658b-4949-b178-ac6530158698/egress-router-binary-copy/0.log" Jun 02 20:35:54.446312 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:54.446281 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5w6p_4271c2ac-658b-4949-b178-ac6530158698/cni-plugins/0.log" Jun 02 20:35:54.479329 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:54.479292 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5w6p_4271c2ac-658b-4949-b178-ac6530158698/bond-cni-plugin/0.log" Jun 02 20:35:54.508350 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:54.508321 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5w6p_4271c2ac-658b-4949-b178-ac6530158698/routeoverride-cni/0.log" Jun 02 20:35:54.533565 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:54.533534 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5w6p_4271c2ac-658b-4949-b178-ac6530158698/whereabouts-cni-bincopy/0.log" Jun 02 20:35:54.558387 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:54.558355 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5w6p_4271c2ac-658b-4949-b178-ac6530158698/whereabouts-cni/0.log" Jun 02 20:35:54.919057 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:54.919020 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wwpfn_c848a2cb-1edf-4aa2-b593-c0f19d847476/network-metrics-daemon/0.log" Jun 02 20:35:54.945609 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:54.945585 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wwpfn_c848a2cb-1edf-4aa2-b593-c0f19d847476/kube-rbac-proxy/0.log" Jun 02 20:35:55.880842 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:55.880762 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-controller/0.log" Jun 02 20:35:55.899926 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:55.899901 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/0.log" Jun 02 20:35:55.911130 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:55.911096 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/ovn-acl-logging/1.log" Jun 02 20:35:55.940273 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:55.940226 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/kube-rbac-proxy-node/0.log" Jun 02 20:35:55.967898 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:55.967864 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 20:35:56.012999 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:56.012958 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/northd/0.log" Jun 02 20:35:56.039808 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:56.039772 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/nbdb/0.log" Jun 02 20:35:56.070196 ip-10-0-129-157 kubenswrapper[2548]: I0602 20:35:56.070167 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-np2n4_fcbbef18-c454-4403-861d-a1cd4cae3ce9/sbdb/0.log"