Jun 02 21:31:06.601842 ip-10-0-128-185 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 02 21:31:06.601856 ip-10-0-128-185 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 02 21:31:06.601865 ip-10-0-128-185 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 02 21:31:06.602258 ip-10-0-128-185 systemd[1]: Failed to start Kubernetes Kubelet. Jun 02 21:31:16.748141 ip-10-0-128-185 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 02 21:31:16.748153 ip-10-0-128-185 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 2073b7f7a31744bca74e0236f64877ac -- Jun 02 21:33:43.421202 ip-10-0-128-185 systemd[1]: Starting Kubernetes Kubelet... Jun 02 21:33:43.778956 ip-10-0-128-185 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 21:33:43.778956 ip-10-0-128-185 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 21:33:43.778956 ip-10-0-128-185 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 21:33:43.778956 ip-10-0-128-185 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 21:33:43.778956 ip-10-0-128-185 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 21:33:43.781013 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.780946 2562 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 21:33:43.786974 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.786959 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:33:43.786974 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.786973 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.786977 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.786980 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.786984 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.786987 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.786990 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.786994 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.786996 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.786999 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787002 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787004 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787007 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787010 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787013 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787015 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787018 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787020 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787023 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787025 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787028 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:33:43.787039 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787030 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787033 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787035 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787038 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787041 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787043 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787046 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787049 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787052 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787055 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787058 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787060 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787062 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787065 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787067 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787069 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787072 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787074 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787077 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787079 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:33:43.787588 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787081 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787084 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787087 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787090 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787092 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787094 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787098 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787102 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787105 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787107 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787109 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787112 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787114 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787116 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787121 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787124 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787127 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787130 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787133 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787136 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:33:43.788384 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787138 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787141 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787144 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787146 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787149 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787151 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787154 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787156 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787158 2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787161 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787163 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787165 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787168 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787170 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787173 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787175 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787177 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787180 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787182 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787184 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:33:43.788915 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787187 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787190 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787192 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787194 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.787197 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788476 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788494 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788502 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788507 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788512 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788517 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788521 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788526 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788531 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788536 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788540 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788544 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788548 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788552 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:33:43.789489 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788557 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788560 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788565 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788569 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788573 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788577 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788582 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788585 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788589 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788593 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788597 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788601 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788605 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788610 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788614 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788619 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788623 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788627 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788634 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788638 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:33:43.789963 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788643 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788646 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788650 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788652 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788655 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788657 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788662 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788664 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788667 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788669 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788673 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788675 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788678 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788681 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788683 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788686 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788688 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788691 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788693 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:33:43.790456 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788696 2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788699 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788702 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788704 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788707 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788709 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788712 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788714 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788717 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788719 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788722 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788725 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788728 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788734 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788738 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788741 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788744 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788747 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788750 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788752 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:33:43.790943 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788755 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788757 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788760 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788762 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788764 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788774 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788777 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788780 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788782 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788785 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788787 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788789 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.788792 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789433 2562 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789443 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789450 2562 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789454 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789458 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789461 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789466 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789470 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 21:33:43.791429 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789474 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789477 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789480 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789484 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789487 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789491 2562 flags.go:64] FLAG: --cgroup-root="" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789493 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789496 2562 flags.go:64] FLAG: --client-ca-file="" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789499 2562 flags.go:64] FLAG: --cloud-config="" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789502 2562 flags.go:64] FLAG: --cloud-provider="external" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789505 2562 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789509 2562 flags.go:64] FLAG: --cluster-domain="" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789512 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789515 2562 flags.go:64] FLAG: --config-dir="" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789518 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789521 2562 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789525 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789534 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789538 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789541 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789544 2562 flags.go:64] FLAG: --contention-profiling="false" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789547 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789549 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789552 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789555 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 21:33:43.791965 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789559 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789562 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789565 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789567 2562 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789570 2562 flags.go:64] FLAG: --enable-server="true" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789573 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789578 2562 flags.go:64] FLAG: --event-burst="100" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789580 2562 flags.go:64] FLAG: --event-qps="50" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789583 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789586 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789590 2562 flags.go:64] FLAG: --eviction-hard="" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789594 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789597 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789600 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789603 2562 flags.go:64] FLAG: --eviction-soft="" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789605 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789608 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789611 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789614 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789616 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789619 2562 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789622 2562 flags.go:64] FLAG: --feature-gates="" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789625 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789628 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789631 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 21:33:43.792589 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789641 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789645 2562 flags.go:64] FLAG: --healthz-port="10248" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789648 2562 flags.go:64] FLAG: --help="false" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789650 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-128-185.ec2.internal" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789653 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789656 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789659 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789662 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789666 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789669 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789671 2562 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789674 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789677 2562 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789680 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789683 2562 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789686 2562 flags.go:64] FLAG: --kube-reserved="" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789688 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789691 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789694 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789697 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789699 2562 flags.go:64] FLAG: --lock-file="" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789702 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789705 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789708 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 21:33:43.793202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789726 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789730 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789733 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789736 2562 flags.go:64] FLAG: --logging-format="text" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789739 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789742 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789745 2562 flags.go:64] FLAG: --manifest-url="" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789748 2562 flags.go:64] FLAG: --manifest-url-header="" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789752 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789760 2562 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789764 2562 flags.go:64] FLAG: --max-pods="110" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789767 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789770 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789773 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789776 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789778 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789781 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789784 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789792 2562 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789795 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789798 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789801 2562 flags.go:64] FLAG: --pod-cidr="" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789804 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789808 2562 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 21:33:43.793778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789811 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789814 2562 flags.go:64] FLAG: --pods-per-core="0" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789817 2562 flags.go:64] FLAG: --port="10250" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789820 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789823 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b301b85207b20a96" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789826 2562 flags.go:64] FLAG: --qos-reserved="" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789829 2562 flags.go:64] FLAG: --read-only-port="10255" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789831 2562 flags.go:64] FLAG: --register-node="true" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789834 2562 flags.go:64] FLAG: --register-schedulable="true" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789838 2562 flags.go:64] FLAG: --register-with-taints="" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789845 2562 flags.go:64] FLAG: --registry-burst="10" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789848 2562 flags.go:64] FLAG: --registry-qps="5" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789850 2562 flags.go:64] FLAG: --reserved-cpus="" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789853 2562 flags.go:64] FLAG: --reserved-memory="" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789856 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789859 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789862 2562 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789879 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789889 2562 flags.go:64] FLAG: --runonce="false" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789892 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789895 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789898 2562 flags.go:64] FLAG: --seccomp-default="false" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789901 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789904 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789907 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789910 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 21:33:43.794396 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789913 2562 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789916 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789918 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789921 2562 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789924 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789926 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789929 2562 flags.go:64] FLAG: --system-cgroups="" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789932 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789938 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789940 2562 flags.go:64] FLAG: --tls-cert-file="" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789943 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789947 2562 flags.go:64] FLAG: --tls-min-version="" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789950 2562 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789952 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789955 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789959 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789962 2562 flags.go:64] FLAG: --v="2" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789970 2562 flags.go:64] FLAG: --version="false" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789974 2562 flags.go:64] FLAG: --vmodule="" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789978 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.789981 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790079 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790084 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790087 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:33:43.795051 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790090 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790099 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790102 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790105 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790107 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790110 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790113 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790116 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790118 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790120 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790123 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790125 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790128 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790130 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790133 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790135 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790138 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790140 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790143 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790145 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:33:43.795678 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790147 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790150 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790152 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790157 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790160 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790162 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790165 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790167 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790169 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790172 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790174 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790177 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790180 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790182 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790190 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790192 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790195 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790197 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790200 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790202 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:33:43.796250 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790205 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790207 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790210 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790212 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790214 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790217 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790219 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790221 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790224 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790226 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790228 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790231 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790233 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790235 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790238 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790242 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790244 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790247 2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790249 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790253 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:33:43.796733 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790256 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790259 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790261 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790264 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790266 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790270 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790273 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790281 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790283 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790287 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790290 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790292 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790295 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790297 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790300 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790302 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790305 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790307 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790310 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790312 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:33:43.797232 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790315 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790317 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.790319 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.790982 2562 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.797219 2562 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.797318 2562 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797367 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797372 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797375 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797379 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797382 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797385 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797390 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797394 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797397 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797400 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:33:43.797797 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797403 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797406 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797409 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797418 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797421 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797423 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797426 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797428 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797431 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797433 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797435 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797438 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797440 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797443 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797445 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797448 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797450 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797453 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797455 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:33:43.798227 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797458 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797461 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797466 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797469 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797472 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797475 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797477 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797480 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797483 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797485 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797488 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797491 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797493 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797496 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797498 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797501 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797503 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797511 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797513 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:33:43.798671 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797516 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797518 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797521 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797523 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797525 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797528 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797530 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797532 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797535 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797537 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797540 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797542 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797544 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797547 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797549 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797552 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797554 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797557 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797559 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797561 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:33:43.799169 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797564 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797566 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797568 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797571 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797573 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797576 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797578 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797580 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797583 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797585 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797588 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797595 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797598 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797601 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797603 2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797606 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797608 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:33:43.799653 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797610 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.797615 2562 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797708 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797712 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797716 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797720 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797722 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797725 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797727 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797730 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797732 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797735 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797737 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797740 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797742 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:33:43.800088 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797745 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797747 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797749 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797752 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797754 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797756 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797759 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797761 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797763 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797766 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797768 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797771 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797774 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797776 2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797780 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797783 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797785 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797788 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797790 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797792 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:33:43.800471 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797795 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797797 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797799 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797802 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797805 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797807 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797810 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797812 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797815 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797817 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797820 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797823 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797825 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797827 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797830 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797832 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797834 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797836 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797839 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797841 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:33:43.801041 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797844 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797846 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797848 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797850 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797853 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797855 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797858 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797860 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797863 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797881 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797883 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797886 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797888 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797891 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797893 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797895 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797898 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797900 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797902 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797905 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:33:43.801550 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797908 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797910 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797912 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797915 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797917 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797920 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797922 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797924 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797927 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797929 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797932 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797934 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:43.797936 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.797941 2562 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.798446 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 21:33:43.802071 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.800827 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 21:33:43.802438 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.801616 2562 server.go:1019] "Starting client certificate rotation" Jun 02 21:33:43.802438 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.801707 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 21:33:43.802438 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.801735 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 21:33:43.820394 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.820381 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 21:33:43.822471 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.822451 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 21:33:43.842296 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.842270 2562 log.go:25] "Validated CRI v1 runtime API" Jun 02 21:33:43.847702 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.847686 2562 log.go:25] "Validated CRI v1 image API" Jun 02 21:33:43.848986 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.848970 2562 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 21:33:43.854202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.854185 2562 fs.go:135] Filesystem UUIDs: map[0bf64946-a0fe-4a48-8b20-7964b028845d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bbdc4c90-5f25-4b3d-b46d-17910d66918e:/dev/nvme0n1p3] Jun 02 21:33:43.854264 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.854202 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 21:33:43.855357 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.855341 2562 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 21:33:43.858236 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.858138 2562 manager.go:217] Machine: {Timestamp:2026-06-02 21:33:43.857656775 +0000 UTC m=+0.340241369 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102396 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2441aef9dc90ab031284d5902bebf6 SystemUUID:ec2441ae-f9dc-90ab-0312-84d5902bebf6 BootID:2073b7f7-a317-44bc-a74e-0236f64877ac Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:86:01:a8:39:47 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:86:01:a8:39:47 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:cb:a4:0d:46:b4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 21:33:43.858766 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.858756 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 21:33:43.858880 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.858855 2562 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 21:33:43.859803 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.859785 2562 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 21:33:43.859947 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.859805 2562 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-185.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 21:33:43.859996 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.859956 2562 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 21:33:43.859996 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.859964 2562 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 21:33:43.859996 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.859971 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 21:33:43.860419 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.860409 2562 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 21:33:43.861333 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.861322 2562 state_mem.go:36] "Initialized new in-memory state store" Jun 02 21:33:43.861430 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.861422 2562 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 21:33:43.863248 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.863239 2562 kubelet.go:491] "Attempting to sync node with API server" Jun 02 21:33:43.863285 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.863253 2562 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 21:33:43.863285 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.863264 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 21:33:43.863285 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.863271 2562 kubelet.go:397] "Adding apiserver pod source" Jun 02 21:33:43.863285 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.863279 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 21:33:43.864108 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.864097 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 21:33:43.864160 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.864115 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 21:33:43.868518 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.868504 2562 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 21:33:43.870090 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.870076 2562 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 21:33:43.872657 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.872639 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 21:33:43.872657 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.872660 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 21:33:43.872793 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.872668 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 21:33:43.872793 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.872681 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 21:33:43.872793 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.872693 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 21:33:43.872793 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.872701 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 21:33:43.872793 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.872710 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 21:33:43.872793 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.872718 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 21:33:43.872793 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.872728 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 21:33:43.872793 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.872742 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 21:33:43.872793 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.872754 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 21:33:43.872793 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.872763 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 21:33:43.874182 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.874169 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 21:33:43.874268 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.874188 2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 21:33:43.880404 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:43.880369 2562 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-185.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 21:33:43.880404 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.880377 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-89gqq" Jun 02 21:33:43.880965 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:43.880938 2562 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 21:33:43.881041 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.880951 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-89gqq" Jun 02 21:33:43.884813 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.884801 2562 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 21:33:43.884891 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.884834 2562 server.go:1295] "Started kubelet" Jun 02 21:33:43.884964 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.884919 2562 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 21:33:43.885025 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.884998 2562 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 21:33:43.885186 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.885166 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 21:33:43.885603 ip-10-0-128-185 systemd[1]: Started Kubernetes Kubelet. Jun 02 21:33:43.886077 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.886061 2562 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 21:33:43.887019 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.887001 2562 server.go:317] "Adding debug handlers to kubelet server" Jun 02 21:33:43.890352 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.890338 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 21:33:43.890463 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.890450 2562 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 21:33:43.890518 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.890465 2562 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 21:33:43.890565 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.890522 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 21:33:43.890620 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.890577 2562 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 21:33:43.891294 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.890835 2562 reconstruct.go:97] "Volume reconstruction finished" Jun 02 21:33:43.891294 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.890853 2562 reconciler.go:26] "Reconciler: start to sync state" Jun 02 21:33:43.891489 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.891472 2562 factory.go:55] Registering systemd factory Jun 02 21:33:43.891545 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.891526 2562 factory.go:223] Registration of the systemd container factory successfully Jun 02 21:33:43.891626 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:43.891609 2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:43.893623 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.893607 2562 factory.go:153] Registering CRI-O factory Jun 02 21:33:43.893723 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.893709 2562 factory.go:223] Registration of the crio container factory successfully Jun 02 21:33:43.893822 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.893800 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 21:33:43.893899 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.893835 2562 factory.go:103] Registering Raw factory Jun 02 21:33:43.893899 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.893880 2562 manager.go:1196] Started watching for new ooms in manager Jun 02 21:33:43.894952 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.894932 2562 manager.go:319] Starting recovery of all containers Jun 02 21:33:43.899338 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.899315 2562 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:33:43.899464 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:43.899443 2562 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 21:33:43.905263 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.905191 2562 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-185.ec2.internal" not found Jun 02 21:33:43.905263 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:43.905203 2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-185.ec2.internal\" not found" node="ip-10-0-128-185.ec2.internal" Jun 02 21:33:43.908669 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.908650 2562 manager.go:324] Recovery completed Jun 02 21:33:43.912821 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.912804 2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:33:43.914683 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.914665 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:33:43.914751 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.914693 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:33:43.914751 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.914702 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:33:43.915226 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.915215 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 21:33:43.915263 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.915226 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 21:33:43.915263 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.915243 2562 state_mem.go:36] "Initialized new in-memory state store" Jun 02 21:33:43.917190 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.917178 2562 policy_none.go:49] "None policy: Start" Jun 02 21:33:43.917250 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.917193 2562 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 21:33:43.917250 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.917202 2562 state_mem.go:35] "Initializing new in-memory state store" Jun 02 21:33:43.922447 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.922432 2562 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-185.ec2.internal" not found Jun 02 21:33:43.954463 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.954451 2562 manager.go:341] "Starting Device Plugin manager" Jun 02 21:33:43.956250 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:43.954481 2562 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 21:33:43.956250 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.954490 2562 server.go:85] "Starting device plugin registration server" Jun 02 21:33:43.956250 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.954669 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 21:33:43.956250 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.954680 2562 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 21:33:43.956250 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.954817 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 21:33:43.956250 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.954926 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 21:33:43.956250 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.954937 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 21:33:43.956250 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:43.955462 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 21:33:43.956250 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:43.955490 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:43.981517 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.981492 2562 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 21:33:43.982707 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.982686 2562 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 21:33:43.982707 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.982707 2562 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 21:33:43.982842 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.982722 2562 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 21:33:43.982842 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.982729 2562 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 21:33:43.982842 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:43.982799 2562 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 21:33:43.985088 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.985068 2562 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:33:43.986320 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:43.986301 2562 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-185.ec2.internal" not found Jun 02 21:33:44.055179 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.055127 2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:33:44.055884 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.055850 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:33:44.055959 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.055896 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:33:44.055959 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.055910 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:33:44.055959 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.055939 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.062172 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.062159 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.062231 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.062178 2562 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-185.ec2.internal\": node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:44.072570 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.072550 2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:44.083693 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.083663 2562 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-185.ec2.internal"] Jun 02 21:33:44.083771 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.083737 2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:33:44.084522 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.084503 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:33:44.084585 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.084529 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:33:44.084585 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.084540 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:33:44.085582 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.085571 2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:33:44.085792 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.085776 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.085851 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.085804 2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:33:44.086208 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.086195 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:33:44.086285 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.086217 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:33:44.086285 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.086228 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:33:44.086380 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.086367 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:33:44.086436 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.086395 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:33:44.086436 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.086410 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:33:44.087187 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.087171 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.087271 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.087197 2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:33:44.087831 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.087814 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:33:44.087831 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.087843 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:33:44.088002 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.087858 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:33:44.091048 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.091033 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aec71be92c8b6b09577cb76075fa990c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-185.ec2.internal\" (UID: \"aec71be92c8b6b09577cb76075fa990c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.091107 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.091060 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2eccb1aaac410d48601679180247b9ca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal\" (UID: \"2eccb1aaac410d48601679180247b9ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.091143 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.091109 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2eccb1aaac410d48601679180247b9ca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal\" (UID: \"2eccb1aaac410d48601679180247b9ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.110461 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.110437 2562 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-185.ec2.internal\" not found" node="ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.114781 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.114765 2562 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-185.ec2.internal\" not found" node="ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.172997 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.172981 2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:44.191225 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.191210 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aec71be92c8b6b09577cb76075fa990c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-185.ec2.internal\" (UID: \"aec71be92c8b6b09577cb76075fa990c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.191284 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.191251 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aec71be92c8b6b09577cb76075fa990c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-185.ec2.internal\" (UID: \"aec71be92c8b6b09577cb76075fa990c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.191284 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.191279 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2eccb1aaac410d48601679180247b9ca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal\" (UID: \"2eccb1aaac410d48601679180247b9ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.191348 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.191295 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2eccb1aaac410d48601679180247b9ca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal\" (UID: \"2eccb1aaac410d48601679180247b9ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.191348 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.191312 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2eccb1aaac410d48601679180247b9ca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal\" (UID: \"2eccb1aaac410d48601679180247b9ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.191348 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.191331 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2eccb1aaac410d48601679180247b9ca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal\" (UID: \"2eccb1aaac410d48601679180247b9ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.273756 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.273734 2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:44.374393 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.374306 2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:44.412660 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.412640 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.417398 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.417378 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-185.ec2.internal" Jun 02 21:33:44.474450 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.474430 2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:44.575281 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.575264 2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:44.675772 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.675716 2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:44.776195 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.776175 2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:44.801636 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.801611 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 21:33:44.802307 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.801759 2562 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 21:33:44.802307 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.801795 2562 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 21:33:44.877355 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.877185 2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:44.879730 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:44.879697 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eccb1aaac410d48601679180247b9ca.slice/crio-64a44989d3d030c6a231e3242c77866b190400adce220131db0a495f66599211 WatchSource:0}: Error finding container 64a44989d3d030c6a231e3242c77866b190400adce220131db0a495f66599211: Status 404 returned error can't find the container with id 64a44989d3d030c6a231e3242c77866b190400adce220131db0a495f66599211 Jun 02 21:33:44.879925 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:44.879904 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec71be92c8b6b09577cb76075fa990c.slice/crio-963efca84d39fda28990b650fa4784c26503c10568978c20366288149c272b78 WatchSource:0}: Error finding container 963efca84d39fda28990b650fa4784c26503c10568978c20366288149c272b78: Status 404 returned error can't find the container with id 963efca84d39fda28990b650fa4784c26503c10568978c20366288149c272b78 Jun 02 21:33:44.884042 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.884028 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:33:44.885691 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.885666 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 21:28:43 +0000 UTC" deadline="2028-02-28 02:15:07.653583996 +0000 UTC" Jun 02 21:33:44.885735 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.885691 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15244h41m22.767894838s" Jun 02 21:33:44.890807 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.890788 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 21:33:44.915054 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.915035 2562 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 21:33:44.935763 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.935722 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-89d67" Jun 02 21:33:44.943896 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.943862 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-89d67" Jun 02 21:33:44.978387 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:44.978361 2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-185.ec2.internal\" not found" Jun 02 21:33:44.985176 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.985140 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" event={"ID":"2eccb1aaac410d48601679180247b9ca","Type":"ContainerStarted","Data":"64a44989d3d030c6a231e3242c77866b190400adce220131db0a495f66599211"} Jun 02 21:33:44.985950 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:44.985931 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-185.ec2.internal" event={"ID":"aec71be92c8b6b09577cb76075fa990c","Type":"ContainerStarted","Data":"963efca84d39fda28990b650fa4784c26503c10568978c20366288149c272b78"} Jun 02 21:33:45.047049 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.047025 2562 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:33:45.091410 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.091384 2562 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" Jun 02 21:33:45.103062 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.103042 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 21:33:45.104442 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.104429 2562 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-185.ec2.internal" Jun 02 21:33:45.113205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.113190 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 21:33:45.450716 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.450679 2562 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:33:45.629912 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.629862 2562 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:33:45.864403 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.864301 2562 apiserver.go:52] "Watching apiserver" Jun 02 21:33:45.871677 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.871654 2562 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 21:33:45.872888 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.872844 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-p58r9","openshift-image-registry/node-ca-8v5xt","openshift-multus/multus-additional-cni-plugins-25w52","openshift-network-diagnostics/network-check-target-vpfh6","openshift-network-operator/iptables-alerter-s5hs9","openshift-ovn-kubernetes/ovnkube-node-z8npt","kube-system/konnectivity-agent-jzc49","kube-system/kube-apiserver-proxy-ip-10-0-128-185.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn","openshift-cluster-node-tuning-operator/tuned-r4cbx","openshift-dns/node-resolver-zvfhn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal","openshift-multus/multus-d85n4"] Jun 02 21:33:45.875196 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.875174 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8v5xt" Jun 02 21:33:45.875303 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.875205 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jzc49" Jun 02 21:33:45.877170 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.877150 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:45.877932 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.877898 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 21:33:45.878071 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.878052 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jbfwk\"" Jun 02 21:33:45.878251 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.878228 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:45.878366 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:45.878300 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:33:45.878366 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.878328 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s5hs9" Jun 02 21:33:45.879190 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.878788 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 21:33:45.879190 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.878801 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 21:33:45.879190 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.878826 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 21:33:45.879190 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.878889 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dbd9d\"" Jun 02 21:33:45.879190 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.878789 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 21:33:45.879497 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.879475 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 21:33:45.879612 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.879596 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.879731 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.879710 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x5bbk\"" Jun 02 21:33:45.879916 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.879898 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 21:33:45.880012 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.879903 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 21:33:45.880362 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.880345 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 21:33:45.880737 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.880693 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 21:33:45.882142 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.882105 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:45.882468 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:45.882441 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:33:45.883254 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.882767 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kfnf4\"" Jun 02 21:33:45.883380 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.883311 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:33:45.883460 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.883367 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 21:33:45.883509 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.883466 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 21:33:45.883556 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.883526 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 21:33:45.883904 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.883812 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 21:33:45.883973 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.883964 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 21:33:45.885012 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.884942 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 21:33:45.885101 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.885073 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 21:33:45.885175 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.885145 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6gcvk\"" Jun 02 21:33:45.885309 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.885294 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 21:33:45.886549 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.886534 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:45.887983 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.887963 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.889044 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.889021 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 21:33:45.889146 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.889067 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 21:33:45.889207 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.889174 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 21:33:45.889262 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.889234 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nn8m7\"" Jun 02 21:33:45.889417 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.889398 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zvfhn" Jun 02 21:33:45.890057 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.890041 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:33:45.890384 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.890362 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-54nlb\"" Jun 02 21:33:45.890469 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.890388 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 21:33:45.891505 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.891486 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rvkzg\"" Jun 02 21:33:45.891590 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.891508 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.891684 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.891652 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 21:33:45.891884 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.891847 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 21:33:45.893392 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.893374 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 21:33:45.893814 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.893796 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8922f\"" Jun 02 21:33:45.900109 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900079 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2vrq\" (UniqueName: \"kubernetes.io/projected/3e99f0a6-186b-4e51-ac27-e394f90e86b1-kube-api-access-z2vrq\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.900238 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-run-netns\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.900238 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-ovnkube-script-lib\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.900238 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900223 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqc8g\" (UniqueName: \"kubernetes.io/projected/b4c54dfc-ecd1-466b-90db-eca67b4780c5-kube-api-access-hqc8g\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:45.900459 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-kubelet-dir\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:45.900459 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900290 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-etc-selinux\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:45.900459 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8e23d364-9727-4c56-b730-501e7f75fd21-etc-tuned\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.900459 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900344 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:45.900459 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900407 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:45.900459 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900421 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-device-dir\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:45.900459 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900442 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-multus-socket-dir-parent\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.900827 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900485 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3e99f0a6-186b-4e51-ac27-e394f90e86b1-multus-daemon-config\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.900827 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900504 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-cni-netd\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.900827 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900519 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-kubernetes\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.900827 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900535 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-var-lib-cni-bin\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.901051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.900962 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-multus-conf-dir\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.901051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901001 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1e9b389-999a-4410-8048-ff2e03df62dc-host\") pod \"node-ca-8v5xt\" (UID: \"c1e9b389-999a-4410-8048-ff2e03df62dc\") " pod="openshift-image-registry/node-ca-8v5xt" Jun 02 21:33:45.901051 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901027 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-cnibin\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:45.901179 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901050 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-sysctl-d\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.901179 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-sysctl-conf\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.901179 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901095 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sgzb\" (UniqueName: \"kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb\") pod \"network-check-target-vpfh6\" (UID: \"301cea8a-d795-4207-8970-4dbec0729108\") " pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:45.901179 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901120 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-run-ovn\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.901179 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901151 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-node-log\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.901179 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901176 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-systemd\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.901431 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901194 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c1e9b389-999a-4410-8048-ff2e03df62dc-serviceca\") pod \"node-ca-8v5xt\" (UID: \"c1e9b389-999a-4410-8048-ff2e03df62dc\") " pod="openshift-image-registry/node-ca-8v5xt" Jun 02 21:33:45.901431 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901227 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-kubelet\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.901431 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.901431 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901284 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js2t4\" (UniqueName: \"kubernetes.io/projected/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-kube-api-access-js2t4\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:45.901431 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901308 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:45.901431 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901321 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-modprobe-d\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.901431 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901334 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-host\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.901431 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901352 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-system-cni-dir\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.901431 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901372 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-cnibin\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.901715 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901428 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-log-socket\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.901715 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e99f0a6-186b-4e51-ac27-e394f90e86b1-cni-binary-copy\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.901715 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-run-k8s-cni-cncf-io\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.901715 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901515 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-hostroot\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.901715 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901544 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-systemd-units\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.901715 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901566 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-slash\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.901715 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901583 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-cni-bin\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.901715 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-os-release\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:45.901715 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901626 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zms4j\" (UniqueName: \"kubernetes.io/projected/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-kube-api-access-zms4j\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:45.901715 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901651 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-etc-openvswitch\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.901715 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901687 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvm64\" (UniqueName: \"kubernetes.io/projected/8e23d364-9727-4c56-b730-501e7f75fd21-kube-api-access-hvm64\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901718 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttdsm\" (UniqueName: \"kubernetes.io/projected/b44e90a0-109c-481c-bb97-0e077753e751-kube-api-access-ttdsm\") pod \"node-resolver-zvfhn\" (UID: \"b44e90a0-109c-481c-bb97-0e077753e751\") " pod="openshift-dns/node-resolver-zvfhn" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901746 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvdgt\" (UniqueName: \"kubernetes.io/projected/c1e9b389-999a-4410-8048-ff2e03df62dc-kube-api-access-pvdgt\") pod \"node-ca-8v5xt\" (UID: \"c1e9b389-999a-4410-8048-ff2e03df62dc\") " pod="openshift-image-registry/node-ca-8v5xt" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901771 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-system-cni-dir\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901796 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-registration-dir\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901841 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-run\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901910 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-run-multus-certs\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cf05a3f2-dd0b-49da-a189-e214dafedee9-konnectivity-ca\") pod \"konnectivity-agent-jzc49\" (UID: \"cf05a3f2-dd0b-49da-a189-e214dafedee9\") " pod="kube-system/konnectivity-agent-jzc49" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.901989 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902014 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n62z\" (UniqueName: \"kubernetes.io/projected/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-kube-api-access-8n62z\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902047 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-lib-modules\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902068 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-var-lib-kubelet\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902084 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-var-lib-cni-multus\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902113 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b-host-slash\") pod \"iptables-alerter-s5hs9\" (UID: \"c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b\") " pod="openshift-network-operator/iptables-alerter-s5hs9" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902130 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-cni-binary-copy\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902143 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-run-openvswitch\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.902185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902159 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902179 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-sys\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902193 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b44e90a0-109c-481c-bb97-0e077753e751-tmp-dir\") pod \"node-resolver-zvfhn\" (UID: \"b44e90a0-109c-481c-bb97-0e077753e751\") " pod="openshift-dns/node-resolver-zvfhn" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b-iptables-alerter-script\") pod \"iptables-alerter-s5hs9\" (UID: \"c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b\") " pod="openshift-network-operator/iptables-alerter-s5hs9" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902225 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhm8q\" (UniqueName: \"kubernetes.io/projected/c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b-kube-api-access-vhm8q\") pod \"iptables-alerter-s5hs9\" (UID: \"c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b\") " pod="openshift-network-operator/iptables-alerter-s5hs9" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-run-systemd\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902264 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-ovnkube-config\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902287 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e23d364-9727-4c56-b730-501e7f75fd21-tmp\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902306 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b44e90a0-109c-481c-bb97-0e077753e751-hosts-file\") pod \"node-resolver-zvfhn\" (UID: \"b44e90a0-109c-481c-bb97-0e077753e751\") " pod="openshift-dns/node-resolver-zvfhn" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902330 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-var-lib-openvswitch\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902365 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-env-overrides\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902411 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-ovn-node-metrics-cert\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902428 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-socket-dir\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902457 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-sysconfig\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902483 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-multus-cni-dir\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902507 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-var-lib-kubelet\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902524 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cf05a3f2-dd0b-49da-a189-e214dafedee9-agent-certs\") pod \"konnectivity-agent-jzc49\" (UID: \"cf05a3f2-dd0b-49da-a189-e214dafedee9\") " pod="kube-system/konnectivity-agent-jzc49" Jun 02 21:33:45.902705 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902543 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-sys-fs\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:45.903255 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902565 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-os-release\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.903255 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-run-netns\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.903255 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.902649 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-etc-kubernetes\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:45.944514 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.944489 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 21:28:44 +0000 UTC" deadline="2028-01-28 17:25:04.962583337 +0000 UTC" Jun 02 21:33:45.944514 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.944512 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14515h51m19.01807374s" Jun 02 21:33:45.991511 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:45.991488 2562 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 21:33:46.003049 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003020 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-sys\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.003154 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003062 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b44e90a0-109c-481c-bb97-0e077753e751-tmp-dir\") pod \"node-resolver-zvfhn\" (UID: \"b44e90a0-109c-481c-bb97-0e077753e751\") " pod="openshift-dns/node-resolver-zvfhn" Jun 02 21:33:46.003154 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003089 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b-iptables-alerter-script\") pod \"iptables-alerter-s5hs9\" (UID: \"c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b\") " pod="openshift-network-operator/iptables-alerter-s5hs9" Jun 02 21:33:46.003154 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003115 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vhm8q\" (UniqueName: \"kubernetes.io/projected/c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b-kube-api-access-vhm8q\") pod \"iptables-alerter-s5hs9\" (UID: \"c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b\") " pod="openshift-network-operator/iptables-alerter-s5hs9" Jun 02 21:33:46.003154 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003139 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-run-systemd\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.003154 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003148 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-sys\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.003365 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003162 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-ovnkube-config\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.003365 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003200 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e23d364-9727-4c56-b730-501e7f75fd21-tmp\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.003365 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003230 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b44e90a0-109c-481c-bb97-0e077753e751-hosts-file\") pod \"node-resolver-zvfhn\" (UID: \"b44e90a0-109c-481c-bb97-0e077753e751\") " pod="openshift-dns/node-resolver-zvfhn" Jun 02 21:33:46.003365 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003237 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-run-systemd\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.003365 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003257 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-var-lib-openvswitch\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.003365 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003284 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-env-overrides\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.003365 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003309 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-ovn-node-metrics-cert\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.003365 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003332 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-socket-dir\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.003365 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003354 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-sysconfig\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003374 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-multus-cni-dir\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003397 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-var-lib-kubelet\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003412 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b44e90a0-109c-481c-bb97-0e077753e751-tmp-dir\") pod \"node-resolver-zvfhn\" (UID: \"b44e90a0-109c-481c-bb97-0e077753e751\") " pod="openshift-dns/node-resolver-zvfhn" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003420 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cf05a3f2-dd0b-49da-a189-e214dafedee9-agent-certs\") pod \"konnectivity-agent-jzc49\" (UID: \"cf05a3f2-dd0b-49da-a189-e214dafedee9\") " pod="kube-system/konnectivity-agent-jzc49" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003445 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-sys-fs\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003469 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-os-release\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003492 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-run-netns\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003516 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-etc-kubernetes\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003525 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003544 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z2vrq\" (UniqueName: \"kubernetes.io/projected/3e99f0a6-186b-4e51-ac27-e394f90e86b1-kube-api-access-z2vrq\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003571 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-run-netns\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003574 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b44e90a0-109c-481c-bb97-0e077753e751-hosts-file\") pod \"node-resolver-zvfhn\" (UID: \"b44e90a0-109c-481c-bb97-0e077753e751\") " pod="openshift-dns/node-resolver-zvfhn" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003595 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-ovnkube-script-lib\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003618 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hqc8g\" (UniqueName: \"kubernetes.io/projected/b4c54dfc-ecd1-466b-90db-eca67b4780c5-kube-api-access-hqc8g\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003641 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-kubelet-dir\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.003664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003666 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-etc-selinux\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003672 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b-iptables-alerter-script\") pod \"iptables-alerter-s5hs9\" (UID: \"c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b\") " pod="openshift-network-operator/iptables-alerter-s5hs9" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003690 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8e23d364-9727-4c56-b730-501e7f75fd21-etc-tuned\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003718 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003725 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-etc-kubernetes\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003766 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003795 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-device-dir\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003811 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-sysconfig\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003822 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-multus-socket-dir-parent\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003849 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3e99f0a6-186b-4e51-ac27-e394f90e86b1-multus-daemon-config\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003893 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-multus-cni-dir\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.003923 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003934 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-cni-netd\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003939 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-multus-socket-dir-parent\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003980 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-var-lib-kubelet\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003984 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-device-dir\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.004005 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs podName:b4c54dfc-ecd1-466b-90db-eca67b4780c5 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:46.503983325 +0000 UTC m=+2.986567944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs") pod "network-metrics-daemon-p58r9" (UID: "b4c54dfc-ecd1-466b-90db-eca67b4780c5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:46.004243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003677 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-ovnkube-config\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004438 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-env-overrides\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004501 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3e99f0a6-186b-4e51-ac27-e394f90e86b1-multus-daemon-config\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004586 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-os-release\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004645 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-sys-fs\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004676 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-ovnkube-script-lib\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004692 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-run-netns\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003896 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-cni-netd\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004740 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-kubernetes\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004744 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004766 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-var-lib-cni-bin\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004791 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-multus-conf-dir\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004815 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1e9b389-999a-4410-8048-ff2e03df62dc-host\") pod \"node-ca-8v5xt\" (UID: \"c1e9b389-999a-4410-8048-ff2e03df62dc\") " pod="openshift-image-registry/node-ca-8v5xt" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004840 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-cnibin\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003530 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-socket-dir\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004889 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-sysctl-d\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004915 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-sysctl-conf\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004932 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-run-netns\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.005205 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004981 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-kubernetes\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.004999 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1e9b389-999a-4410-8048-ff2e03df62dc-host\") pod \"node-ca-8v5xt\" (UID: \"c1e9b389-999a-4410-8048-ff2e03df62dc\") " pod="openshift-image-registry/node-ca-8v5xt" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005014 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-kubelet-dir\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005023 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-cnibin\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005043 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-var-lib-cni-bin\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005066 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-etc-selinux\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005080 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-multus-conf-dir\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005095 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgzb\" (UniqueName: \"kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb\") pod \"network-check-target-vpfh6\" (UID: \"301cea8a-d795-4207-8970-4dbec0729108\") " pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005137 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-run-ovn\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005153 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-sysctl-d\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005169 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-node-log\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005189 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-sysctl-conf\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005193 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-run-ovn\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005194 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-systemd\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005232 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c1e9b389-999a-4410-8048-ff2e03df62dc-serviceca\") pod \"node-ca-8v5xt\" (UID: \"c1e9b389-999a-4410-8048-ff2e03df62dc\") " pod="openshift-image-registry/node-ca-8v5xt" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005239 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-systemd\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005236 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-node-log\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005261 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-kubelet\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.005979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005290 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-kubelet\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005289 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005323 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.003527 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-var-lib-openvswitch\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005328 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-js2t4\" (UniqueName: \"kubernetes.io/projected/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-kube-api-access-js2t4\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005371 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005398 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-modprobe-d\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005423 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-host\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005448 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-system-cni-dir\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005475 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-cnibin\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005502 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-log-socket\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005529 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e99f0a6-186b-4e51-ac27-e394f90e86b1-cni-binary-copy\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005544 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-host\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005577 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c1e9b389-999a-4410-8048-ff2e03df62dc-serviceca\") pod \"node-ca-8v5xt\" (UID: \"c1e9b389-999a-4410-8048-ff2e03df62dc\") " pod="openshift-image-registry/node-ca-8v5xt" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005596 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-cnibin\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005634 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-log-socket\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005647 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005681 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-run-k8s-cni-cncf-io\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.006767 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005693 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-etc-modprobe-d\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005683 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-system-cni-dir\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005712 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-hostroot\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005739 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-systemd-units\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005756 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-hostroot\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005759 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-slash\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005781 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-slash\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005713 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-run-k8s-cni-cncf-io\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005798 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-cni-bin\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005825 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-systemd-units\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005827 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-os-release\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005850 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-cni-bin\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005857 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zms4j\" (UniqueName: \"kubernetes.io/projected/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-kube-api-access-zms4j\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005907 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-etc-openvswitch\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005917 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-os-release\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005934 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hvm64\" (UniqueName: \"kubernetes.io/projected/8e23d364-9727-4c56-b730-501e7f75fd21-kube-api-access-hvm64\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005954 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-etc-openvswitch\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005961 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ttdsm\" (UniqueName: \"kubernetes.io/projected/b44e90a0-109c-481c-bb97-0e077753e751-kube-api-access-ttdsm\") pod \"node-resolver-zvfhn\" (UID: \"b44e90a0-109c-481c-bb97-0e077753e751\") " pod="openshift-dns/node-resolver-zvfhn" Jun 02 21:33:46.007521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005975 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e99f0a6-186b-4e51-ac27-e394f90e86b1-cni-binary-copy\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.005988 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pvdgt\" (UniqueName: \"kubernetes.io/projected/c1e9b389-999a-4410-8048-ff2e03df62dc-kube-api-access-pvdgt\") pod \"node-ca-8v5xt\" (UID: \"c1e9b389-999a-4410-8048-ff2e03df62dc\") " pod="openshift-image-registry/node-ca-8v5xt" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006015 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-system-cni-dir\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006043 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-registration-dir\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006066 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-run\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006091 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-run-multus-certs\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006118 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cf05a3f2-dd0b-49da-a189-e214dafedee9-konnectivity-ca\") pod \"konnectivity-agent-jzc49\" (UID: \"cf05a3f2-dd0b-49da-a189-e214dafedee9\") " pod="kube-system/konnectivity-agent-jzc49" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006142 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006168 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8n62z\" (UniqueName: \"kubernetes.io/projected/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-kube-api-access-8n62z\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006174 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-system-cni-dir\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006191 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-lib-modules\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006212 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-var-lib-kubelet\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006239 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-var-lib-cni-multus\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006254 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-registration-dir\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006264 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b-host-slash\") pod \"iptables-alerter-s5hs9\" (UID: \"c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b\") " pod="openshift-network-operator/iptables-alerter-s5hs9" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006289 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-cni-binary-copy\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006343 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-run-openvswitch\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.008247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006347 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-run-multus-certs\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006371 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006395 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3e99f0a6-186b-4e51-ac27-e394f90e86b1-host-var-lib-cni-multus\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006443 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b-host-slash\") pod \"iptables-alerter-s5hs9\" (UID: \"c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b\") " pod="openshift-network-operator/iptables-alerter-s5hs9" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006480 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006485 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-lib-modules\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006313 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-run\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006537 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e23d364-9727-4c56-b730-501e7f75fd21-var-lib-kubelet\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006543 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-run-openvswitch\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.006639 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.007007 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-cni-binary-copy\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.007774 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e23d364-9727-4c56-b730-501e7f75fd21-tmp\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.007927 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-ovn-node-metrics-cert\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.008844 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.007951 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cf05a3f2-dd0b-49da-a189-e214dafedee9-agent-certs\") pod \"konnectivity-agent-jzc49\" (UID: \"cf05a3f2-dd0b-49da-a189-e214dafedee9\") " pod="kube-system/konnectivity-agent-jzc49" Jun 02 21:33:46.009438 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.009110 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8e23d364-9727-4c56-b730-501e7f75fd21-etc-tuned\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.013176 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.012855 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2vrq\" (UniqueName: \"kubernetes.io/projected/3e99f0a6-186b-4e51-ac27-e394f90e86b1-kube-api-access-z2vrq\") pod \"multus-d85n4\" (UID: \"3e99f0a6-186b-4e51-ac27-e394f90e86b1\") " pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.014087 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.014066 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvm64\" (UniqueName: \"kubernetes.io/projected/8e23d364-9727-4c56-b730-501e7f75fd21-kube-api-access-hvm64\") pod \"tuned-r4cbx\" (UID: \"8e23d364-9727-4c56-b730-501e7f75fd21\") " pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.015003 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.014945 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cf05a3f2-dd0b-49da-a189-e214dafedee9-konnectivity-ca\") pod \"konnectivity-agent-jzc49\" (UID: \"cf05a3f2-dd0b-49da-a189-e214dafedee9\") " pod="kube-system/konnectivity-agent-jzc49" Jun 02 21:33:46.020494 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.018935 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:33:46.020494 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.018959 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:33:46.020494 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.018972 2562 projected.go:194] Error preparing data for projected volume kube-api-access-4sgzb for pod openshift-network-diagnostics/network-check-target-vpfh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:46.020494 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.019027 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb podName:301cea8a-d795-4207-8970-4dbec0729108 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:46.519011671 +0000 UTC m=+3.001596268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4sgzb" (UniqueName: "kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb") pod "network-check-target-vpfh6" (UID: "301cea8a-d795-4207-8970-4dbec0729108") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:46.020765 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.020533 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhm8q\" (UniqueName: \"kubernetes.io/projected/c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b-kube-api-access-vhm8q\") pod \"iptables-alerter-s5hs9\" (UID: \"c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b\") " pod="openshift-network-operator/iptables-alerter-s5hs9" Jun 02 21:33:46.020765 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.020597 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqc8g\" (UniqueName: \"kubernetes.io/projected/b4c54dfc-ecd1-466b-90db-eca67b4780c5-kube-api-access-hqc8g\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:46.022257 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.020981 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js2t4\" (UniqueName: \"kubernetes.io/projected/bf49242d-6557-46b3-a8c0-bfc7a27a3a16-kube-api-access-js2t4\") pod \"aws-ebs-csi-driver-node-62rjn\" (UID: \"bf49242d-6557-46b3-a8c0-bfc7a27a3a16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.022257 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.021023 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvdgt\" (UniqueName: \"kubernetes.io/projected/c1e9b389-999a-4410-8048-ff2e03df62dc-kube-api-access-pvdgt\") pod \"node-ca-8v5xt\" (UID: \"c1e9b389-999a-4410-8048-ff2e03df62dc\") " pod="openshift-image-registry/node-ca-8v5xt" Jun 02 21:33:46.022257 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.021040 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zms4j\" (UniqueName: \"kubernetes.io/projected/3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d-kube-api-access-zms4j\") pod \"multus-additional-cni-plugins-25w52\" (UID: \"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d\") " pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.022257 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.022147 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttdsm\" (UniqueName: \"kubernetes.io/projected/b44e90a0-109c-481c-bb97-0e077753e751-kube-api-access-ttdsm\") pod \"node-resolver-zvfhn\" (UID: \"b44e90a0-109c-481c-bb97-0e077753e751\") " pod="openshift-dns/node-resolver-zvfhn" Jun 02 21:33:46.022967 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.022949 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n62z\" (UniqueName: \"kubernetes.io/projected/042b9156-fdec-4499-9bcc-d1a4f1f24aaf-kube-api-access-8n62z\") pod \"ovnkube-node-z8npt\" (UID: \"042b9156-fdec-4499-9bcc-d1a4f1f24aaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.189233 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.189203 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8v5xt" Jun 02 21:33:46.196146 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.196117 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jzc49" Jun 02 21:33:46.205017 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.204991 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-25w52" Jun 02 21:33:46.210676 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.210659 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s5hs9" Jun 02 21:33:46.215250 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.215233 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:33:46.221830 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.221813 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" Jun 02 21:33:46.228431 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.228409 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" Jun 02 21:33:46.236104 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.236072 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zvfhn" Jun 02 21:33:46.241752 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.241730 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d85n4" Jun 02 21:33:46.385148 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.385123 2562 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:33:46.420572 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:46.420542 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e9b389_999a_4410_8048_ff2e03df62dc.slice/crio-6dcc323959e517d0c5b3f7da8e5d91bde2bcdd22f3e542524cda99fb46f03fde WatchSource:0}: Error finding container 6dcc323959e517d0c5b3f7da8e5d91bde2bcdd22f3e542524cda99fb46f03fde: Status 404 returned error can't find the container with id 6dcc323959e517d0c5b3f7da8e5d91bde2bcdd22f3e542524cda99fb46f03fde Jun 02 21:33:46.421535 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:46.421508 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e99f0a6_186b_4e51_ac27_e394f90e86b1.slice/crio-b0980e8d6219a940a48afe18fb28eb3533ee0064a327ea72ce36b9a039ade844 WatchSource:0}: Error finding container b0980e8d6219a940a48afe18fb28eb3533ee0064a327ea72ce36b9a039ade844: Status 404 returned error can't find the container with id b0980e8d6219a940a48afe18fb28eb3533ee0064a327ea72ce36b9a039ade844 Jun 02 21:33:46.425321 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:46.425295 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e23d364_9727_4c56_b730_501e7f75fd21.slice/crio-4b8058de36f90b4d8f8f12ef3fc6fbb59d5232cbd798837cd0a0381335645739 WatchSource:0}: Error finding container 4b8058de36f90b4d8f8f12ef3fc6fbb59d5232cbd798837cd0a0381335645739: Status 404 returned error can't find the container with id 4b8058de36f90b4d8f8f12ef3fc6fbb59d5232cbd798837cd0a0381335645739 Jun 02 21:33:46.426441 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:46.426413 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod042b9156_fdec_4499_9bcc_d1a4f1f24aaf.slice/crio-2bafc0fb81260eefda4d6a60c0d8b3ade00f18f8dcdfe7c574fc8ad26e3ab8c8 WatchSource:0}: Error finding container 2bafc0fb81260eefda4d6a60c0d8b3ade00f18f8dcdfe7c574fc8ad26e3ab8c8: Status 404 returned error can't find the container with id 2bafc0fb81260eefda4d6a60c0d8b3ade00f18f8dcdfe7c574fc8ad26e3ab8c8 Jun 02 21:33:46.427115 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:46.427094 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb44e90a0_109c_481c_bb97_0e077753e751.slice/crio-7dd65bc0e82648a694baebc3b1243625c1465003d10931383d48abc071b21aeb WatchSource:0}: Error finding container 7dd65bc0e82648a694baebc3b1243625c1465003d10931383d48abc071b21aeb: Status 404 returned error can't find the container with id 7dd65bc0e82648a694baebc3b1243625c1465003d10931383d48abc071b21aeb Jun 02 21:33:46.427942 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:46.427922 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf05a3f2_dd0b_49da_a189_e214dafedee9.slice/crio-736b35f972f3304ca2dec32894228e21bf4e59b8037514c1e8a593d940973c36 WatchSource:0}: Error finding container 736b35f972f3304ca2dec32894228e21bf4e59b8037514c1e8a593d940973c36: Status 404 returned error can't find the container with id 736b35f972f3304ca2dec32894228e21bf4e59b8037514c1e8a593d940973c36 Jun 02 21:33:46.429475 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:46.429157 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf49242d_6557_46b3_a8c0_bfc7a27a3a16.slice/crio-d5fd7badf6b049a561fb1c62b70c286e9953907af694ccfed00c0c36c5937e44 WatchSource:0}: Error finding container d5fd7badf6b049a561fb1c62b70c286e9953907af694ccfed00c0c36c5937e44: Status 404 returned error can't find the container with id d5fd7badf6b049a561fb1c62b70c286e9953907af694ccfed00c0c36c5937e44 Jun 02 21:33:46.436518 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:33:46.436488 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c087a4_ad5e_40c4_8b41_4ce0bdbf762b.slice/crio-85e55327fd1ccf9a5fd10f06ee4fd21d86a91ed609e3a2659a4f89cf364eeb8b WatchSource:0}: Error finding container 85e55327fd1ccf9a5fd10f06ee4fd21d86a91ed609e3a2659a4f89cf364eeb8b: Status 404 returned error can't find the container with id 85e55327fd1ccf9a5fd10f06ee4fd21d86a91ed609e3a2659a4f89cf364eeb8b Jun 02 21:33:46.510015 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.509987 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:46.510165 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.510148 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:46.510316 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.510301 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs podName:b4c54dfc-ecd1-466b-90db-eca67b4780c5 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:47.510287518 +0000 UTC m=+3.992872103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs") pod "network-metrics-daemon-p58r9" (UID: "b4c54dfc-ecd1-466b-90db-eca67b4780c5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:46.610534 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.610506 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgzb\" (UniqueName: \"kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb\") pod \"network-check-target-vpfh6\" (UID: \"301cea8a-d795-4207-8970-4dbec0729108\") " pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:46.610669 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.610622 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:33:46.610669 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.610640 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:33:46.610669 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.610649 2562 projected.go:194] Error preparing data for projected volume kube-api-access-4sgzb for pod openshift-network-diagnostics/network-check-target-vpfh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:46.610774 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.610691 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb podName:301cea8a-d795-4207-8970-4dbec0729108 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:47.610678616 +0000 UTC m=+4.093263196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4sgzb" (UniqueName: "kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb") pod "network-check-target-vpfh6" (UID: "301cea8a-d795-4207-8970-4dbec0729108") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:46.945717 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.945640 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 21:28:44 +0000 UTC" deadline="2027-12-08 10:20:25.286831686 +0000 UTC" Jun 02 21:33:46.945717 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.945675 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13284h46m38.341160399s" Jun 02 21:33:46.983223 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:46.983196 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:46.983354 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:46.983330 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:33:47.009494 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.009387 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" event={"ID":"bf49242d-6557-46b3-a8c0-bfc7a27a3a16","Type":"ContainerStarted","Data":"d5fd7badf6b049a561fb1c62b70c286e9953907af694ccfed00c0c36c5937e44"} Jun 02 21:33:47.018135 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.018109 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8v5xt" event={"ID":"c1e9b389-999a-4410-8048-ff2e03df62dc","Type":"ContainerStarted","Data":"6dcc323959e517d0c5b3f7da8e5d91bde2bcdd22f3e542524cda99fb46f03fde"} Jun 02 21:33:47.034220 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.034192 2562 generic.go:358] "Generic (PLEG): container finished" podID="2eccb1aaac410d48601679180247b9ca" containerID="0569801c9a2619002458556fb1b30fe45ee6f81f36da578030566762fcd9ffd7" exitCode=0 Jun 02 21:33:47.034333 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.034262 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" event={"ID":"2eccb1aaac410d48601679180247b9ca","Type":"ContainerDied","Data":"0569801c9a2619002458556fb1b30fe45ee6f81f36da578030566762fcd9ffd7"} Jun 02 21:33:47.045114 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.045087 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s5hs9" event={"ID":"c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b","Type":"ContainerStarted","Data":"85e55327fd1ccf9a5fd10f06ee4fd21d86a91ed609e3a2659a4f89cf364eeb8b"} Jun 02 21:33:47.061448 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.061423 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25w52" event={"ID":"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d","Type":"ContainerStarted","Data":"906d94a7286af0ab76a4883d6d419ea2391fff7277bd7dc131125a4541150524"} Jun 02 21:33:47.073149 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.073126 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zvfhn" event={"ID":"b44e90a0-109c-481c-bb97-0e077753e751","Type":"ContainerStarted","Data":"7dd65bc0e82648a694baebc3b1243625c1465003d10931383d48abc071b21aeb"} Jun 02 21:33:47.079586 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.079562 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" event={"ID":"042b9156-fdec-4499-9bcc-d1a4f1f24aaf","Type":"ContainerStarted","Data":"2bafc0fb81260eefda4d6a60c0d8b3ade00f18f8dcdfe7c574fc8ad26e3ab8c8"} Jun 02 21:33:47.086016 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.085993 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" event={"ID":"8e23d364-9727-4c56-b730-501e7f75fd21","Type":"ContainerStarted","Data":"4b8058de36f90b4d8f8f12ef3fc6fbb59d5232cbd798837cd0a0381335645739"} Jun 02 21:33:47.118826 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.118629 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d85n4" event={"ID":"3e99f0a6-186b-4e51-ac27-e394f90e86b1","Type":"ContainerStarted","Data":"b0980e8d6219a940a48afe18fb28eb3533ee0064a327ea72ce36b9a039ade844"} Jun 02 21:33:47.131132 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.131106 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-185.ec2.internal" event={"ID":"aec71be92c8b6b09577cb76075fa990c","Type":"ContainerStarted","Data":"7bc79fc79fea5af9f809be7c87e9f10082f7fd732dca2392ac054237ef554fa1"} Jun 02 21:33:47.144800 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.144766 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jzc49" event={"ID":"cf05a3f2-dd0b-49da-a189-e214dafedee9","Type":"ContainerStarted","Data":"736b35f972f3304ca2dec32894228e21bf4e59b8037514c1e8a593d940973c36"} Jun 02 21:33:47.524141 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.524026 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:47.524306 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:47.524236 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:47.524364 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:47.524323 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs podName:b4c54dfc-ecd1-466b-90db-eca67b4780c5 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:49.524302546 +0000 UTC m=+6.006887155 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs") pod "network-metrics-daemon-p58r9" (UID: "b4c54dfc-ecd1-466b-90db-eca67b4780c5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:47.624550 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.624516 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgzb\" (UniqueName: \"kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb\") pod \"network-check-target-vpfh6\" (UID: \"301cea8a-d795-4207-8970-4dbec0729108\") " pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:47.624719 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:47.624703 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:33:47.624776 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:47.624724 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:33:47.624776 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:47.624737 2562 projected.go:194] Error preparing data for projected volume kube-api-access-4sgzb for pod openshift-network-diagnostics/network-check-target-vpfh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:47.624909 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:47.624792 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb podName:301cea8a-d795-4207-8970-4dbec0729108 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:49.624772663 +0000 UTC m=+6.107357249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4sgzb" (UniqueName: "kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb") pod "network-check-target-vpfh6" (UID: "301cea8a-d795-4207-8970-4dbec0729108") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:47.983237 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:47.983203 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:47.983690 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:47.983337 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:33:48.167665 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:48.167572 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" event={"ID":"2eccb1aaac410d48601679180247b9ca","Type":"ContainerStarted","Data":"cd6b3a43f359dd2eeb149236dd2743a6f51d8b88f916971c464fcabe0d988823"} Jun 02 21:33:48.181803 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:48.180975 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-185.ec2.internal" podStartSLOduration=3.180960205 podStartE2EDuration="3.180960205s" podCreationTimestamp="2026-06-02 21:33:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:33:47.144929186 +0000 UTC m=+3.627513792" watchObservedRunningTime="2026-06-02 21:33:48.180960205 +0000 UTC m=+4.663544810" Jun 02 21:33:48.983841 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:48.983350 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:48.983841 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:48.983484 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:33:49.540930 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:49.540892 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:49.541093 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:49.541004 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:49.541093 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:49.541074 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs podName:b4c54dfc-ecd1-466b-90db-eca67b4780c5 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:53.541056135 +0000 UTC m=+10.023640728 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs") pod "network-metrics-daemon-p58r9" (UID: "b4c54dfc-ecd1-466b-90db-eca67b4780c5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:49.642329 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:49.642047 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgzb\" (UniqueName: \"kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb\") pod \"network-check-target-vpfh6\" (UID: \"301cea8a-d795-4207-8970-4dbec0729108\") " pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:49.642329 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:49.642219 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:33:49.642329 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:49.642239 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:33:49.642329 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:49.642253 2562 projected.go:194] Error preparing data for projected volume kube-api-access-4sgzb for pod openshift-network-diagnostics/network-check-target-vpfh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:49.642329 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:49.642311 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb podName:301cea8a-d795-4207-8970-4dbec0729108 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:53.642292396 +0000 UTC m=+10.124876982 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4sgzb" (UniqueName: "kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb") pod "network-check-target-vpfh6" (UID: "301cea8a-d795-4207-8970-4dbec0729108") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:49.985932 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:49.985460 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:49.985932 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:49.985579 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:33:50.983574 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:50.983542 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:50.983754 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:50.983692 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:33:51.984460 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:51.983976 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:51.984460 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:51.984104 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:33:52.983616 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:52.983573 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:52.983806 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:52.983706 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:33:53.574337 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:53.574302 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:53.574789 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:53.574473 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:53.574789 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:53.574548 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs podName:b4c54dfc-ecd1-466b-90db-eca67b4780c5 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:01.574528795 +0000 UTC m=+18.057113383 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs") pod "network-metrics-daemon-p58r9" (UID: "b4c54dfc-ecd1-466b-90db-eca67b4780c5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:53.676228 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:53.675578 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgzb\" (UniqueName: \"kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb\") pod \"network-check-target-vpfh6\" (UID: \"301cea8a-d795-4207-8970-4dbec0729108\") " pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:53.676228 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:53.675751 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:33:53.676228 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:53.675769 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:33:53.676228 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:53.675781 2562 projected.go:194] Error preparing data for projected volume kube-api-access-4sgzb for pod openshift-network-diagnostics/network-check-target-vpfh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:53.676228 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:53.675836 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb podName:301cea8a-d795-4207-8970-4dbec0729108 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:01.675815647 +0000 UTC m=+18.158400242 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4sgzb" (UniqueName: "kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb") pod "network-check-target-vpfh6" (UID: "301cea8a-d795-4207-8970-4dbec0729108") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:53.985418 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:53.984943 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:53.985418 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:53.985051 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:33:54.983311 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:54.983277 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:54.983701 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:54.983395 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:33:55.983762 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:55.983564 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:55.984272 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:55.983819 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:33:56.983336 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:56.983309 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:56.983513 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:56.983434 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:33:57.983360 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:57.983328 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:57.983929 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:57.983468 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:33:58.983886 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:58.983843 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:33:58.984311 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:58.983981 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:33:59.983797 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:33:59.983764 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:33:59.983996 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:33:59.983895 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:34:00.983823 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:00.983786 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:00.984077 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:00.983943 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:34:01.635585 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:01.635493 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:01.635746 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:01.635630 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:34:01.635746 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:01.635708 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs podName:b4c54dfc-ecd1-466b-90db-eca67b4780c5 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:17.635690653 +0000 UTC m=+34.118275235 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs") pod "network-metrics-daemon-p58r9" (UID: "b4c54dfc-ecd1-466b-90db-eca67b4780c5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:34:01.736147 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:01.736112 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgzb\" (UniqueName: \"kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb\") pod \"network-check-target-vpfh6\" (UID: \"301cea8a-d795-4207-8970-4dbec0729108\") " pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:01.736297 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:01.736258 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:34:01.736297 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:01.736277 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:34:01.736297 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:01.736289 2562 projected.go:194] Error preparing data for projected volume kube-api-access-4sgzb for pod openshift-network-diagnostics/network-check-target-vpfh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:34:01.736413 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:01.736339 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb podName:301cea8a-d795-4207-8970-4dbec0729108 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:17.736327176 +0000 UTC m=+34.218911757 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4sgzb" (UniqueName: "kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb") pod "network-check-target-vpfh6" (UID: "301cea8a-d795-4207-8970-4dbec0729108") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:34:01.983488 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:01.983458 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:01.983647 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:01.983575 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:34:02.983022 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:02.982978 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:02.983449 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:02.983105 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:34:03.984196 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:03.984159 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:03.984584 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:03.984248 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:34:04.983558 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:04.983529 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:04.983700 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:04.983630 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:34:05.982988 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:05.982962 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:05.983665 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:05.983090 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:34:06.186561 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.186531 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" event={"ID":"8e23d364-9727-4c56-b730-501e7f75fd21","Type":"ContainerStarted","Data":"a61d9b15a9d0a9480dfdcacc66d56ed18aa2cfc181790277ab89fa4840facbfb"} Jun 02 21:34:06.188184 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.188153 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d85n4" event={"ID":"3e99f0a6-186b-4e51-ac27-e394f90e86b1","Type":"ContainerStarted","Data":"c297754cea0d284fdbc0005b569ad7776a30399e87a2468f8b4945bd4ccdddcd"} Jun 02 21:34:06.189824 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.189794 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jzc49" event={"ID":"cf05a3f2-dd0b-49da-a189-e214dafedee9","Type":"ContainerStarted","Data":"83af309737cc7544d63a0e7776ca40403d67177d622925e012ac47271d924cf8"} Jun 02 21:34:06.191382 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.191360 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" event={"ID":"bf49242d-6557-46b3-a8c0-bfc7a27a3a16","Type":"ContainerStarted","Data":"55d41c572e507553d103261c357e139105f4df4639b88ad2b45df8fb48ddf6ba"} Jun 02 21:34:06.192588 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.192564 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8v5xt" event={"ID":"c1e9b389-999a-4410-8048-ff2e03df62dc","Type":"ContainerStarted","Data":"4c062dc91f4477a83b43a21558b3580845244170007ac30113a8614b1f61eaae"} Jun 02 21:34:06.194299 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.194272 2562 generic.go:358] "Generic (PLEG): container finished" podID="3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d" containerID="e90d06514e4cd6135b80fc6e37b55af0c0229f838854d0a432de9d2b4e32cc81" exitCode=0 Jun 02 21:34:06.194428 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.194353 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25w52" event={"ID":"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d","Type":"ContainerDied","Data":"e90d06514e4cd6135b80fc6e37b55af0c0229f838854d0a432de9d2b4e32cc81"} Jun 02 21:34:06.195842 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.195819 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zvfhn" event={"ID":"b44e90a0-109c-481c-bb97-0e077753e751","Type":"ContainerStarted","Data":"a1291e10878d7b98a454f72c042027cfa86ed518393b457921f21c4d60d5daa2"} Jun 02 21:34:06.199120 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.199094 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" event={"ID":"042b9156-fdec-4499-9bcc-d1a4f1f24aaf","Type":"ContainerStarted","Data":"c4dc9c5b90aed8261b3ab7b4f59827536006f054ff367b486e6ffc57cfceb021"} Jun 02 21:34:06.199207 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.199130 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" event={"ID":"042b9156-fdec-4499-9bcc-d1a4f1f24aaf","Type":"ContainerStarted","Data":"104a2cda71dc39c46a5629a51fd54e13ef503f8d8b6d5d62340bcc5d35e2afdf"} Jun 02 21:34:06.199207 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.199147 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" event={"ID":"042b9156-fdec-4499-9bcc-d1a4f1f24aaf","Type":"ContainerStarted","Data":"809b43f2537df47b8e7a06f9edca74895ad9c81b762483cd87d9ecb49328ac28"} Jun 02 21:34:06.199207 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.199162 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" event={"ID":"042b9156-fdec-4499-9bcc-d1a4f1f24aaf","Type":"ContainerStarted","Data":"30078e51727b9c4ede66c508ae3be0703487c6942b77aafe7a35332ab14d3c61"} Jun 02 21:34:06.199207 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.199176 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" event={"ID":"042b9156-fdec-4499-9bcc-d1a4f1f24aaf","Type":"ContainerStarted","Data":"dd20997b4fddd61c330bf47dfdb0b147b35f9d4e0ad374d096cd74277371df18"} Jun 02 21:34:06.204198 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.204160 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r4cbx" podStartSLOduration=3.3021339530000002 podStartE2EDuration="22.204145003s" podCreationTimestamp="2026-06-02 21:33:44 +0000 UTC" firstStartedPulling="2026-06-02 21:33:46.427269959 +0000 UTC m=+2.909854545" lastFinishedPulling="2026-06-02 21:34:05.329281007 +0000 UTC m=+21.811865595" observedRunningTime="2026-06-02 21:34:06.203884333 +0000 UTC m=+22.686468928" watchObservedRunningTime="2026-06-02 21:34:06.204145003 +0000 UTC m=+22.686729613" Jun 02 21:34:06.204304 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.204285 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-185.ec2.internal" podStartSLOduration=21.204281016 podStartE2EDuration="21.204281016s" podCreationTimestamp="2026-06-02 21:33:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:33:48.181786622 +0000 UTC m=+4.664371227" watchObservedRunningTime="2026-06-02 21:34:06.204281016 +0000 UTC m=+22.686865625" Jun 02 21:34:06.221861 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.221821 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-d85n4" podStartSLOduration=3.286620454 podStartE2EDuration="22.221807941s" podCreationTimestamp="2026-06-02 21:33:44 +0000 UTC" firstStartedPulling="2026-06-02 21:33:46.424222762 +0000 UTC m=+2.906807346" lastFinishedPulling="2026-06-02 21:34:05.359410251 +0000 UTC m=+21.841994833" observedRunningTime="2026-06-02 21:34:06.221515886 +0000 UTC m=+22.704100489" watchObservedRunningTime="2026-06-02 21:34:06.221807941 +0000 UTC m=+22.704392551" Jun 02 21:34:06.239322 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.239271 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8v5xt" podStartSLOduration=7.379288221 podStartE2EDuration="22.23925604s" podCreationTimestamp="2026-06-02 21:33:44 +0000 UTC" firstStartedPulling="2026-06-02 21:33:46.42417092 +0000 UTC m=+2.906755518" lastFinishedPulling="2026-06-02 21:34:01.28413875 +0000 UTC m=+17.766723337" observedRunningTime="2026-06-02 21:34:06.236045716 +0000 UTC m=+22.718630320" watchObservedRunningTime="2026-06-02 21:34:06.23925604 +0000 UTC m=+22.721840644" Jun 02 21:34:06.277546 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.277497 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jzc49" podStartSLOduration=3.379825766 podStartE2EDuration="22.27748667s" podCreationTimestamp="2026-06-02 21:33:44 +0000 UTC" firstStartedPulling="2026-06-02 21:33:46.429643298 +0000 UTC m=+2.912227883" lastFinishedPulling="2026-06-02 21:34:05.327304196 +0000 UTC m=+21.809888787" observedRunningTime="2026-06-02 21:34:06.255133197 +0000 UTC m=+22.737717802" watchObservedRunningTime="2026-06-02 21:34:06.27748667 +0000 UTC m=+22.760071273" Jun 02 21:34:06.674881 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.674680 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 21:34:06.966711 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.966602 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T21:34:06.674881697Z","UUID":"085212ac-0277-40b6-9d99-5d2092066b4c","Handler":null,"Name":"","Endpoint":""} Jun 02 21:34:06.968605 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.968581 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 21:34:06.968733 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.968613 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 21:34:06.983604 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:06.983571 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:06.983954 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:06.983681 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:34:07.203270 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:07.203233 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" event={"ID":"bf49242d-6557-46b3-a8c0-bfc7a27a3a16","Type":"ContainerStarted","Data":"cca306f0452cee5a0068ddfca87428330a25008d6b5860a426e90d2d87ded620"} Jun 02 21:34:07.204840 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:07.204809 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s5hs9" event={"ID":"c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b","Type":"ContainerStarted","Data":"aa714f443544d7be5af72ed528455cbbce9631118fb468ceab1d10867286d9b3"} Jun 02 21:34:07.208791 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:07.208318 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" event={"ID":"042b9156-fdec-4499-9bcc-d1a4f1f24aaf","Type":"ContainerStarted","Data":"aa838d1ea4d32d3f1fd008a30e205a9a721f94fa75ced29573e75f92f27e69aa"} Jun 02 21:34:07.218556 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:07.218478 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-zvfhn" podStartSLOduration=4.747293889 podStartE2EDuration="23.218467077s" podCreationTimestamp="2026-06-02 21:33:44 +0000 UTC" firstStartedPulling="2026-06-02 21:33:46.428815617 +0000 UTC m=+2.911400216" lastFinishedPulling="2026-06-02 21:34:04.89998881 +0000 UTC m=+21.382573404" observedRunningTime="2026-06-02 21:34:06.29363093 +0000 UTC m=+22.776215534" watchObservedRunningTime="2026-06-02 21:34:07.218467077 +0000 UTC m=+23.701051670" Jun 02 21:34:07.218855 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:07.218832 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-s5hs9" podStartSLOduration=4.329024361 podStartE2EDuration="23.218823924s" podCreationTimestamp="2026-06-02 21:33:44 +0000 UTC" firstStartedPulling="2026-06-02 21:33:46.439296536 +0000 UTC m=+2.921881122" lastFinishedPulling="2026-06-02 21:34:05.329096094 +0000 UTC m=+21.811680685" observedRunningTime="2026-06-02 21:34:07.218598395 +0000 UTC m=+23.701183002" watchObservedRunningTime="2026-06-02 21:34:07.218823924 +0000 UTC m=+23.701408529" Jun 02 21:34:07.983809 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:07.983772 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:07.984328 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:07.983917 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:34:08.212471 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:08.212432 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" event={"ID":"bf49242d-6557-46b3-a8c0-bfc7a27a3a16","Type":"ContainerStarted","Data":"9a52f3ac917c0bb1f0be3ac6708f59b686e3ea655082f64b1e899c162e41ea64"} Jun 02 21:34:08.232087 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:08.232040 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62rjn" podStartSLOduration=3.125877307 podStartE2EDuration="24.232019529s" podCreationTimestamp="2026-06-02 21:33:44 +0000 UTC" firstStartedPulling="2026-06-02 21:33:46.435997431 +0000 UTC m=+2.918582016" lastFinishedPulling="2026-06-02 21:34:07.542139657 +0000 UTC m=+24.024724238" observedRunningTime="2026-06-02 21:34:08.231905338 +0000 UTC m=+24.714489943" watchObservedRunningTime="2026-06-02 21:34:08.232019529 +0000 UTC m=+24.714604132" Jun 02 21:34:08.983111 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:08.983080 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:08.983289 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:08.983195 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:34:09.217980 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:09.217925 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" event={"ID":"042b9156-fdec-4499-9bcc-d1a4f1f24aaf","Type":"ContainerStarted","Data":"f0fe1430834cb147e10353bd80df02071fafea38a1eb06f72dd8797cbdf7fae6"} Jun 02 21:34:09.348274 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:09.348201 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jzc49" Jun 02 21:34:09.348894 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:09.348857 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jzc49" Jun 02 21:34:09.984070 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:09.984035 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:09.984248 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:09.984160 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:34:10.220230 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:10.220209 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jzc49" Jun 02 21:34:10.220814 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:10.220797 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jzc49" Jun 02 21:34:10.983996 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:10.983791 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:10.984178 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:10.984081 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:34:11.223079 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:11.223049 2562 generic.go:358] "Generic (PLEG): container finished" podID="3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d" containerID="d5744708df4f33b72c4bb904ee4676271925f4fea0bf56a055d2f0268c916275" exitCode=0 Jun 02 21:34:11.223517 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:11.223122 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25w52" event={"ID":"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d","Type":"ContainerDied","Data":"d5744708df4f33b72c4bb904ee4676271925f4fea0bf56a055d2f0268c916275"} Jun 02 21:34:11.226493 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:11.226467 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" event={"ID":"042b9156-fdec-4499-9bcc-d1a4f1f24aaf","Type":"ContainerStarted","Data":"f3cd7061a9b7b3dbf161000e435c3b86cd791e53b083580831a19b0f1b29fa42"} Jun 02 21:34:11.226933 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:11.226901 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:34:11.242640 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:11.242588 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:34:11.273581 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:11.273536 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" podStartSLOduration=8.119833841 podStartE2EDuration="27.273520242s" podCreationTimestamp="2026-06-02 21:33:44 +0000 UTC" firstStartedPulling="2026-06-02 21:33:46.428219718 +0000 UTC m=+2.910804301" lastFinishedPulling="2026-06-02 21:34:05.581906107 +0000 UTC m=+22.064490702" observedRunningTime="2026-06-02 21:34:11.271832831 +0000 UTC m=+27.754417445" watchObservedRunningTime="2026-06-02 21:34:11.273520242 +0000 UTC m=+27.756104844" Jun 02 21:34:11.983448 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:11.983420 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:11.983572 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:11.983546 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:34:12.229376 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:12.229342 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:34:12.229376 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:12.229382 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:34:12.245123 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:12.244987 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:34:12.315060 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:12.315032 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p58r9"] Jun 02 21:34:12.315207 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:12.315190 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:12.315337 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:12.315318 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:34:12.318668 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:12.318643 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vpfh6"] Jun 02 21:34:12.318762 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:12.318725 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:12.318839 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:12.318819 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:34:13.232477 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:13.232441 2562 generic.go:358] "Generic (PLEG): container finished" podID="3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d" containerID="8053861dd5972bd7f251c34193b89d6d3310360ac354639ef638b42f966a8d64" exitCode=0 Jun 02 21:34:13.233052 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:13.232534 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25w52" event={"ID":"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d","Type":"ContainerDied","Data":"8053861dd5972bd7f251c34193b89d6d3310360ac354639ef638b42f966a8d64"} Jun 02 21:34:13.983713 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:13.983689 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:13.983820 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:13.983773 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:34:13.983928 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:13.983852 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:13.984029 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:13.983997 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:34:14.236422 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:14.236396 2562 generic.go:358] "Generic (PLEG): container finished" podID="3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d" containerID="05e2915ccaa6469d6ce181122368fcbc49e15768b43d96047b450e5d2c701074" exitCode=0 Jun 02 21:34:14.236822 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:14.236485 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25w52" event={"ID":"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d","Type":"ContainerDied","Data":"05e2915ccaa6469d6ce181122368fcbc49e15768b43d96047b450e5d2c701074"} Jun 02 21:34:15.983798 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:15.983770 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:15.984505 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:15.983898 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:34:15.984505 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:15.983937 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:15.984505 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:15.984075 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:34:17.658492 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:17.658452 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:17.659062 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:17.658580 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:34:17.659062 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:17.658645 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs podName:b4c54dfc-ecd1-466b-90db-eca67b4780c5 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:49.658628742 +0000 UTC m=+66.141213322 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs") pod "network-metrics-daemon-p58r9" (UID: "b4c54dfc-ecd1-466b-90db-eca67b4780c5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:34:17.759632 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:17.759605 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgzb\" (UniqueName: \"kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb\") pod \"network-check-target-vpfh6\" (UID: \"301cea8a-d795-4207-8970-4dbec0729108\") " pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:17.759773 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:17.759717 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:34:17.759773 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:17.759730 2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:34:17.759773 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:17.759738 2562 projected.go:194] Error preparing data for projected volume kube-api-access-4sgzb for pod openshift-network-diagnostics/network-check-target-vpfh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:34:17.759918 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:17.759787 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb podName:301cea8a-d795-4207-8970-4dbec0729108 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:49.759774937 +0000 UTC m=+66.242359517 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4sgzb" (UniqueName: "kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb") pod "network-check-target-vpfh6" (UID: "301cea8a-d795-4207-8970-4dbec0729108") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:34:17.984219 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:17.983959 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:17.984393 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:17.983987 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:17.984393 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:17.984338 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:34:17.984393 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:17.984372 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpfh6" podUID="301cea8a-d795-4207-8970-4dbec0729108" Jun 02 21:34:18.419249 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.419174 2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-185.ec2.internal" event="NodeReady" Jun 02 21:34:18.419409 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.419277 2562 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 21:34:18.466718 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.466677 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bs2gf"] Jun 02 21:34:18.497400 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.497375 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vccz8"] Jun 02 21:34:18.497607 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.497585 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:18.500704 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.500681 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 21:34:18.500908 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.500888 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 21:34:18.500984 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.500909 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gs96v\"" Jun 02 21:34:18.511381 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.511354 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bs2gf"] Jun 02 21:34:18.511381 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.511378 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vccz8"] Jun 02 21:34:18.511535 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.511467 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:34:18.516126 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.515216 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 21:34:18.516126 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.515401 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2lz6l\"" Jun 02 21:34:18.516126 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.515812 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 21:34:18.516126 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.515848 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 21:34:18.666292 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.666256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg2w8\" (UniqueName: \"kubernetes.io/projected/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-kube-api-access-qg2w8\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:34:18.666657 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.666302 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:34:18.666657 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.666412 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:18.666657 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.666451 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-tmp-dir\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:18.666657 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.666497 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-config-volume\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:18.666657 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.666516 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq6pw\" (UniqueName: \"kubernetes.io/projected/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-kube-api-access-dq6pw\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:18.767573 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.767525 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:18.767769 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.767750 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-tmp-dir\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:18.767915 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.767787 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-config-volume\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:18.767915 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.767806 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dq6pw\" (UniqueName: \"kubernetes.io/projected/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-kube-api-access-dq6pw\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:18.767915 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:18.767840 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:18.767915 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.767850 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qg2w8\" (UniqueName: \"kubernetes.io/projected/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-kube-api-access-qg2w8\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:34:18.767915 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:18.767909 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls podName:d51d5fac-3da4-4fb8-a84d-6c9e748a91af nodeName:}" failed. No retries permitted until 2026-06-02 21:34:19.267892879 +0000 UTC m=+35.750477461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls") pod "dns-default-bs2gf" (UID: "d51d5fac-3da4-4fb8-a84d-6c9e748a91af") : secret "dns-default-metrics-tls" not found Jun 02 21:34:18.768122 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.767926 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:34:18.768122 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:18.768001 2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:18.768122 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:18.768034 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert podName:68e8e581-ba13-4f0c-8ffe-21bde0160fb7 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:19.268026119 +0000 UTC m=+35.750610700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert") pod "ingress-canary-vccz8" (UID: "68e8e581-ba13-4f0c-8ffe-21bde0160fb7") : secret "canary-serving-cert" not found Jun 02 21:34:18.768480 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.768457 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-tmp-dir\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:18.768554 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.768542 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-config-volume\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:18.778744 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.778720 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq6pw\" (UniqueName: \"kubernetes.io/projected/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-kube-api-access-dq6pw\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:18.778846 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:18.778789 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg2w8\" (UniqueName: \"kubernetes.io/projected/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-kube-api-access-qg2w8\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:34:19.271651 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:19.271615 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:34:19.271822 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:19.271691 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:19.271822 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:19.271786 2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:19.271822 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:19.271808 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:19.271965 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:19.271860 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert podName:68e8e581-ba13-4f0c-8ffe-21bde0160fb7 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:20.271840508 +0000 UTC m=+36.754425090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert") pod "ingress-canary-vccz8" (UID: "68e8e581-ba13-4f0c-8ffe-21bde0160fb7") : secret "canary-serving-cert" not found Jun 02 21:34:19.271965 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:19.271895 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls podName:d51d5fac-3da4-4fb8-a84d-6c9e748a91af nodeName:}" failed. No retries permitted until 2026-06-02 21:34:20.271882887 +0000 UTC m=+36.754467470 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls") pod "dns-default-bs2gf" (UID: "d51d5fac-3da4-4fb8-a84d-6c9e748a91af") : secret "dns-default-metrics-tls" not found Jun 02 21:34:19.984060 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:19.984014 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:19.984444 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:19.984014 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:19.986733 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:19.986712 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 21:34:19.988403 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:19.988376 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4s2f5\"" Jun 02 21:34:19.988515 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:19.988380 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 21:34:19.988515 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:19.988428 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-txpc4\"" Jun 02 21:34:19.988515 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:19.988436 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 21:34:20.280890 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:20.280818 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:34:20.280890 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:20.280887 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:20.281022 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:20.280951 2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:20.281022 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:20.280979 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:20.281022 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:20.281010 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert podName:68e8e581-ba13-4f0c-8ffe-21bde0160fb7 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:22.280994274 +0000 UTC m=+38.763578854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert") pod "ingress-canary-vccz8" (UID: "68e8e581-ba13-4f0c-8ffe-21bde0160fb7") : secret "canary-serving-cert" not found Jun 02 21:34:20.281117 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:20.281026 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls podName:d51d5fac-3da4-4fb8-a84d-6c9e748a91af nodeName:}" failed. No retries permitted until 2026-06-02 21:34:22.281018904 +0000 UTC m=+38.763603485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls") pod "dns-default-bs2gf" (UID: "d51d5fac-3da4-4fb8-a84d-6c9e748a91af") : secret "dns-default-metrics-tls" not found Jun 02 21:34:21.254978 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:21.254950 2562 generic.go:358] "Generic (PLEG): container finished" podID="3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d" containerID="7e4d06f3641ad9df1a907f4bad99a1944c4b624501d00e023c4fde3026aceb84" exitCode=0 Jun 02 21:34:21.255318 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:21.255001 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25w52" event={"ID":"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d","Type":"ContainerDied","Data":"7e4d06f3641ad9df1a907f4bad99a1944c4b624501d00e023c4fde3026aceb84"} Jun 02 21:34:22.259118 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:22.259085 2562 generic.go:358] "Generic (PLEG): container finished" podID="3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d" containerID="8657136f819485cc600a0b15640bd63d45a6e13ce92b939ba4ad9bb9b909acd3" exitCode=0 Jun 02 21:34:22.259452 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:22.259145 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25w52" event={"ID":"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d","Type":"ContainerDied","Data":"8657136f819485cc600a0b15640bd63d45a6e13ce92b939ba4ad9bb9b909acd3"} Jun 02 21:34:22.294836 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:22.294814 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:34:22.294949 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:22.294902 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:22.294949 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:22.294922 2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:22.295052 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:22.294978 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert podName:68e8e581-ba13-4f0c-8ffe-21bde0160fb7 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:26.294956873 +0000 UTC m=+42.777541457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert") pod "ingress-canary-vccz8" (UID: "68e8e581-ba13-4f0c-8ffe-21bde0160fb7") : secret "canary-serving-cert" not found Jun 02 21:34:22.295052 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:22.295036 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:22.295121 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:22.295087 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls podName:d51d5fac-3da4-4fb8-a84d-6c9e748a91af nodeName:}" failed. No retries permitted until 2026-06-02 21:34:26.295074715 +0000 UTC m=+42.777659297 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls") pod "dns-default-bs2gf" (UID: "d51d5fac-3da4-4fb8-a84d-6c9e748a91af") : secret "dns-default-metrics-tls" not found Jun 02 21:34:23.264656 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:23.264514 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25w52" event={"ID":"3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d","Type":"ContainerStarted","Data":"293fd84689905c3d40ef6a155542bf4404682c4436236d0261b38ba321593eb5"} Jun 02 21:34:23.291040 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:23.290993 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-25w52" podStartSLOduration=5.285837085 podStartE2EDuration="39.290979586s" podCreationTimestamp="2026-06-02 21:33:44 +0000 UTC" firstStartedPulling="2026-06-02 21:33:46.436171124 +0000 UTC m=+2.918755706" lastFinishedPulling="2026-06-02 21:34:20.441313613 +0000 UTC m=+36.923898207" observedRunningTime="2026-06-02 21:34:23.289525164 +0000 UTC m=+39.772109764" watchObservedRunningTime="2026-06-02 21:34:23.290979586 +0000 UTC m=+39.773564180" Jun 02 21:34:26.319498 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:26.319460 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:26.319960 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:26.319526 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:34:26.319960 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:26.319613 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:26.319960 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:26.319648 2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:26.319960 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:26.319689 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls podName:d51d5fac-3da4-4fb8-a84d-6c9e748a91af nodeName:}" failed. No retries permitted until 2026-06-02 21:34:34.319672794 +0000 UTC m=+50.802257376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls") pod "dns-default-bs2gf" (UID: "d51d5fac-3da4-4fb8-a84d-6c9e748a91af") : secret "dns-default-metrics-tls" not found Jun 02 21:34:26.319960 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:26.319713 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert podName:68e8e581-ba13-4f0c-8ffe-21bde0160fb7 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:34.319695339 +0000 UTC m=+50.802279922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert") pod "ingress-canary-vccz8" (UID: "68e8e581-ba13-4f0c-8ffe-21bde0160fb7") : secret "canary-serving-cert" not found Jun 02 21:34:34.368497 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:34.368464 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:34:34.369024 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:34.368515 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:34.369024 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:34.368598 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:34.369024 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:34.368603 2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:34.369024 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:34.368647 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls podName:d51d5fac-3da4-4fb8-a84d-6c9e748a91af nodeName:}" failed. No retries permitted until 2026-06-02 21:34:50.368633928 +0000 UTC m=+66.851218510 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls") pod "dns-default-bs2gf" (UID: "d51d5fac-3da4-4fb8-a84d-6c9e748a91af") : secret "dns-default-metrics-tls" not found Jun 02 21:34:34.369024 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:34.368668 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert podName:68e8e581-ba13-4f0c-8ffe-21bde0160fb7 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:50.36865351 +0000 UTC m=+66.851238092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert") pod "ingress-canary-vccz8" (UID: "68e8e581-ba13-4f0c-8ffe-21bde0160fb7") : secret "canary-serving-cert" not found Jun 02 21:34:44.249008 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:44.248983 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8npt" Jun 02 21:34:49.665919 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:49.665862 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:34:49.668720 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:49.668700 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 21:34:49.676745 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:49.676728 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 21:34:49.676810 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:49.676800 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs podName:b4c54dfc-ecd1-466b-90db-eca67b4780c5 nodeName:}" failed. No retries permitted until 2026-06-02 21:35:53.676781672 +0000 UTC m=+130.159366258 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs") pod "network-metrics-daemon-p58r9" (UID: "b4c54dfc-ecd1-466b-90db-eca67b4780c5") : secret "metrics-daemon-secret" not found Jun 02 21:34:49.766951 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:49.766922 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgzb\" (UniqueName: \"kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb\") pod \"network-check-target-vpfh6\" (UID: \"301cea8a-d795-4207-8970-4dbec0729108\") " pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:49.769549 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:49.769536 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 21:34:49.779597 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:49.779579 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 21:34:49.791787 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:49.791767 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sgzb\" (UniqueName: \"kubernetes.io/projected/301cea8a-d795-4207-8970-4dbec0729108-kube-api-access-4sgzb\") pod \"network-check-target-vpfh6\" (UID: \"301cea8a-d795-4207-8970-4dbec0729108\") " pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:49.998683 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:49.998665 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-txpc4\"" Jun 02 21:34:50.006815 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:50.006799 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:50.132743 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:50.132718 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vpfh6"] Jun 02 21:34:50.136209 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:34:50.136180 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod301cea8a_d795_4207_8970_4dbec0729108.slice/crio-363e77e09d789da139bf9b56b0c4d72c07151d8144d78c04603415f7830eefef WatchSource:0}: Error finding container 363e77e09d789da139bf9b56b0c4d72c07151d8144d78c04603415f7830eefef: Status 404 returned error can't find the container with id 363e77e09d789da139bf9b56b0c4d72c07151d8144d78c04603415f7830eefef Jun 02 21:34:50.317255 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:50.317162 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vpfh6" event={"ID":"301cea8a-d795-4207-8970-4dbec0729108","Type":"ContainerStarted","Data":"363e77e09d789da139bf9b56b0c4d72c07151d8144d78c04603415f7830eefef"} Jun 02 21:34:50.371087 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:50.371050 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:34:50.371221 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:50.371111 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:34:50.371221 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:50.371193 2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:50.371221 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:50.371194 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:50.371330 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:50.371252 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert podName:68e8e581-ba13-4f0c-8ffe-21bde0160fb7 nodeName:}" failed. No retries permitted until 2026-06-02 21:35:22.371234435 +0000 UTC m=+98.853819016 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert") pod "ingress-canary-vccz8" (UID: "68e8e581-ba13-4f0c-8ffe-21bde0160fb7") : secret "canary-serving-cert" not found Jun 02 21:34:50.371330 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:34:50.371265 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls podName:d51d5fac-3da4-4fb8-a84d-6c9e748a91af nodeName:}" failed. No retries permitted until 2026-06-02 21:35:22.37125952 +0000 UTC m=+98.853844101 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls") pod "dns-default-bs2gf" (UID: "d51d5fac-3da4-4fb8-a84d-6c9e748a91af") : secret "dns-default-metrics-tls" not found Jun 02 21:34:53.324642 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:53.324605 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vpfh6" event={"ID":"301cea8a-d795-4207-8970-4dbec0729108","Type":"ContainerStarted","Data":"e5e3356e147a485684d1bc5afe0f84061ffad96c316799b3b28de255cc950582"} Jun 02 21:34:53.325016 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:53.324726 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:34:53.338456 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:34:53.338409 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vpfh6" podStartSLOduration=66.744474034 podStartE2EDuration="1m9.338397204s" podCreationTimestamp="2026-06-02 21:33:44 +0000 UTC" firstStartedPulling="2026-06-02 21:34:50.137851423 +0000 UTC m=+66.620436004" lastFinishedPulling="2026-06-02 21:34:52.731774594 +0000 UTC m=+69.214359174" observedRunningTime="2026-06-02 21:34:53.338149971 +0000 UTC m=+69.820734576" watchObservedRunningTime="2026-06-02 21:34:53.338397204 +0000 UTC m=+69.820981801" Jun 02 21:35:22.374920 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:22.374882 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:35:22.375348 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:22.374935 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:35:22.375348 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:22.375022 2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:35:22.375348 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:22.375025 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:35:22.375348 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:22.375073 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert podName:68e8e581-ba13-4f0c-8ffe-21bde0160fb7 nodeName:}" failed. No retries permitted until 2026-06-02 21:36:26.375058271 +0000 UTC m=+162.857642853 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert") pod "ingress-canary-vccz8" (UID: "68e8e581-ba13-4f0c-8ffe-21bde0160fb7") : secret "canary-serving-cert" not found Jun 02 21:35:22.375348 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:22.375098 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls podName:d51d5fac-3da4-4fb8-a84d-6c9e748a91af nodeName:}" failed. No retries permitted until 2026-06-02 21:36:26.375079072 +0000 UTC m=+162.857663654 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls") pod "dns-default-bs2gf" (UID: "d51d5fac-3da4-4fb8-a84d-6c9e748a91af") : secret "dns-default-metrics-tls" not found Jun 02 21:35:24.328891 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:24.328838 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vpfh6" Jun 02 21:35:49.758883 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.758843 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r"] Jun 02 21:35:49.762697 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.762665 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:49.763573 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.763536 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-86dd499c55-wbghk"] Jun 02 21:35:49.765195 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.765162 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 02 21:35:49.765341 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.765312 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hf2zj\"" Jun 02 21:35:49.766445 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.766426 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 21:35:49.766680 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.766663 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 21:35:49.767180 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.767164 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.768918 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.768897 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 02 21:35:49.769328 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.769308 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 02 21:35:49.769808 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.769775 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-p4d5w\"" Jun 02 21:35:49.769956 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.769944 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 21:35:49.770441 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.770147 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 02 21:35:49.770441 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.770164 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 02 21:35:49.770441 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.770225 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 02 21:35:49.770441 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.770309 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 21:35:49.774472 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.774437 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r"] Jun 02 21:35:49.780848 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.780826 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-86dd499c55-wbghk"] Jun 02 21:35:49.844650 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.844631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4jzg\" (UniqueName: \"kubernetes.io/projected/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-kube-api-access-j4jzg\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.844755 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.844667 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:49.844755 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.844688 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-stats-auth\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.844755 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.844708 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-default-certificate\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.844755 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.844734 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.844925 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.844799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.844925 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.844845 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhwt6\" (UniqueName: \"kubernetes.io/projected/e6f2d34a-e816-4c95-9068-359e6d8f43ff-kube-api-access-xhwt6\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:49.844925 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.844908 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e6f2d34a-e816-4c95-9068-359e6d8f43ff-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:49.946088 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.946061 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j4jzg\" (UniqueName: \"kubernetes.io/projected/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-kube-api-access-j4jzg\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.946176 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.946100 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:49.946176 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.946119 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-stats-auth\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.946176 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.946143 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-default-certificate\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.946176 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.946168 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.946330 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.946204 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.946330 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:49.946217 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 21:35:49.946330 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.946253 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xhwt6\" (UniqueName: \"kubernetes.io/projected/e6f2d34a-e816-4c95-9068-359e6d8f43ff-kube-api-access-xhwt6\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:49.946330 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:49.946278 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls podName:e6f2d34a-e816-4c95-9068-359e6d8f43ff nodeName:}" failed. No retries permitted until 2026-06-02 21:35:50.446257818 +0000 UTC m=+126.928842400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-wdn2r" (UID: "e6f2d34a-e816-4c95-9068-359e6d8f43ff") : secret "cluster-monitoring-operator-tls" not found Jun 02 21:35:49.946330 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.946321 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e6f2d34a-e816-4c95-9068-359e6d8f43ff-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:49.946565 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:49.946367 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle podName:37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce nodeName:}" failed. No retries permitted until 2026-06-02 21:35:50.446343672 +0000 UTC m=+126.928928255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle") pod "router-default-86dd499c55-wbghk" (UID: "37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce") : configmap references non-existent config key: service-ca.crt Jun 02 21:35:49.946648 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:49.946628 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 21:35:49.946724 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:49.946710 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs podName:37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce nodeName:}" failed. No retries permitted until 2026-06-02 21:35:50.446692362 +0000 UTC m=+126.929276959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs") pod "router-default-86dd499c55-wbghk" (UID: "37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce") : secret "router-metrics-certs-default" not found Jun 02 21:35:49.946992 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.946973 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e6f2d34a-e816-4c95-9068-359e6d8f43ff-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:49.948946 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.948920 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-default-certificate\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.949034 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.948955 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-stats-auth\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:49.954848 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.954817 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhwt6\" (UniqueName: \"kubernetes.io/projected/e6f2d34a-e816-4c95-9068-359e6d8f43ff-kube-api-access-xhwt6\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:49.955537 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:49.955513 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4jzg\" (UniqueName: \"kubernetes.io/projected/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-kube-api-access-j4jzg\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:50.450804 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:50.450775 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:50.450993 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:50.450823 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:50.450993 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:50.450850 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:50.450993 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:50.450944 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 21:35:50.450993 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:50.450962 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 21:35:50.450993 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:50.450981 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle podName:37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce nodeName:}" failed. No retries permitted until 2026-06-02 21:35:51.450963527 +0000 UTC m=+127.933548108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle") pod "router-default-86dd499c55-wbghk" (UID: "37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce") : configmap references non-existent config key: service-ca.crt Jun 02 21:35:50.451229 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:50.451011 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs podName:37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce nodeName:}" failed. No retries permitted until 2026-06-02 21:35:51.450996683 +0000 UTC m=+127.933581264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs") pod "router-default-86dd499c55-wbghk" (UID: "37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce") : secret "router-metrics-certs-default" not found Jun 02 21:35:50.451229 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:50.451040 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls podName:e6f2d34a-e816-4c95-9068-359e6d8f43ff nodeName:}" failed. No retries permitted until 2026-06-02 21:35:51.451022299 +0000 UTC m=+127.933606880 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-wdn2r" (UID: "e6f2d34a-e816-4c95-9068-359e6d8f43ff") : secret "cluster-monitoring-operator-tls" not found Jun 02 21:35:51.456214 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:51.456175 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:51.456638 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:51.456229 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:51.456638 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:51.456268 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:51.456638 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:51.456336 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 21:35:51.456638 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:51.456384 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle podName:37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce nodeName:}" failed. No retries permitted until 2026-06-02 21:35:53.456364397 +0000 UTC m=+129.938948978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle") pod "router-default-86dd499c55-wbghk" (UID: "37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce") : configmap references non-existent config key: service-ca.crt Jun 02 21:35:51.456638 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:51.456396 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 21:35:51.456638 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:51.456409 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls podName:e6f2d34a-e816-4c95-9068-359e6d8f43ff nodeName:}" failed. No retries permitted until 2026-06-02 21:35:53.456400625 +0000 UTC m=+129.938985206 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-wdn2r" (UID: "e6f2d34a-e816-4c95-9068-359e6d8f43ff") : secret "cluster-monitoring-operator-tls" not found Jun 02 21:35:51.456638 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:51.456446 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs podName:37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce nodeName:}" failed. No retries permitted until 2026-06-02 21:35:53.456429427 +0000 UTC m=+129.939014025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs") pod "router-default-86dd499c55-wbghk" (UID: "37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce") : secret "router-metrics-certs-default" not found Jun 02 21:35:53.470141 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:53.470102 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:53.470141 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:53.470140 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:53.470671 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:53.470169 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:53.470671 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:53.470248 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 21:35:53.470671 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:53.470314 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls podName:e6f2d34a-e816-4c95-9068-359e6d8f43ff nodeName:}" failed. No retries permitted until 2026-06-02 21:35:57.470296615 +0000 UTC m=+133.952881197 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-wdn2r" (UID: "e6f2d34a-e816-4c95-9068-359e6d8f43ff") : secret "cluster-monitoring-operator-tls" not found Jun 02 21:35:53.470671 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:53.470319 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 21:35:53.470671 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:53.470329 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle podName:37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce nodeName:}" failed. No retries permitted until 2026-06-02 21:35:57.47032206 +0000 UTC m=+133.952906642 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle") pod "router-default-86dd499c55-wbghk" (UID: "37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce") : configmap references non-existent config key: service-ca.crt Jun 02 21:35:53.470671 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:53.470357 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs podName:37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce nodeName:}" failed. No retries permitted until 2026-06-02 21:35:57.470343397 +0000 UTC m=+133.952927977 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs") pod "router-default-86dd499c55-wbghk" (UID: "37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce") : secret "router-metrics-certs-default" not found Jun 02 21:35:53.772235 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:53.772151 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:35:53.772379 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:53.772263 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 21:35:53.772379 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:53.772311 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs podName:b4c54dfc-ecd1-466b-90db-eca67b4780c5 nodeName:}" failed. No retries permitted until 2026-06-02 21:37:55.772297498 +0000 UTC m=+252.254882078 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs") pod "network-metrics-daemon-p58r9" (UID: "b4c54dfc-ecd1-466b-90db-eca67b4780c5") : secret "metrics-daemon-secret" not found Jun 02 21:35:55.730884 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.730823 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-qwjj5"] Jun 02 21:35:55.733619 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.733598 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:55.736182 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.736123 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 02 21:35:55.736182 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.736148 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:35:55.736405 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.736179 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 02 21:35:55.736405 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.736239 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kfmnd\"" Jun 02 21:35:55.737238 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.737219 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 02 21:35:55.744157 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.744128 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-qwjj5"] Jun 02 21:35:55.745632 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.745581 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 02 21:35:55.785179 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.785144 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t745x\" (UniqueName: \"kubernetes.io/projected/6c26e207-3abb-40b7-9d56-b3f48d9d3ba4-kube-api-access-t745x\") pod \"console-operator-86d7c6cc77-qwjj5\" (UID: \"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4\") " pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:55.785307 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.785285 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c26e207-3abb-40b7-9d56-b3f48d9d3ba4-trusted-ca\") pod \"console-operator-86d7c6cc77-qwjj5\" (UID: \"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4\") " pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:55.785373 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.785326 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c26e207-3abb-40b7-9d56-b3f48d9d3ba4-serving-cert\") pod \"console-operator-86d7c6cc77-qwjj5\" (UID: \"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4\") " pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:55.785373 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.785364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c26e207-3abb-40b7-9d56-b3f48d9d3ba4-config\") pod \"console-operator-86d7c6cc77-qwjj5\" (UID: \"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4\") " pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:55.871611 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.871586 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zvfhn_b44e90a0-109c-481c-bb97-0e077753e751/dns-node-resolver/0.log" Jun 02 21:35:55.886493 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.886473 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c26e207-3abb-40b7-9d56-b3f48d9d3ba4-config\") pod \"console-operator-86d7c6cc77-qwjj5\" (UID: \"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4\") " pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:55.886555 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.886505 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t745x\" (UniqueName: \"kubernetes.io/projected/6c26e207-3abb-40b7-9d56-b3f48d9d3ba4-kube-api-access-t745x\") pod \"console-operator-86d7c6cc77-qwjj5\" (UID: \"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4\") " pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:55.886614 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.886598 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c26e207-3abb-40b7-9d56-b3f48d9d3ba4-trusted-ca\") pod \"console-operator-86d7c6cc77-qwjj5\" (UID: \"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4\") " pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:55.886649 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.886636 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c26e207-3abb-40b7-9d56-b3f48d9d3ba4-serving-cert\") pod \"console-operator-86d7c6cc77-qwjj5\" (UID: \"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4\") " pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:55.887235 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.887205 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c26e207-3abb-40b7-9d56-b3f48d9d3ba4-config\") pod \"console-operator-86d7c6cc77-qwjj5\" (UID: \"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4\") " pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:55.887346 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.887329 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c26e207-3abb-40b7-9d56-b3f48d9d3ba4-trusted-ca\") pod \"console-operator-86d7c6cc77-qwjj5\" (UID: \"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4\") " pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:55.889047 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.889032 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c26e207-3abb-40b7-9d56-b3f48d9d3ba4-serving-cert\") pod \"console-operator-86d7c6cc77-qwjj5\" (UID: \"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4\") " pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:55.894347 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:55.894326 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t745x\" (UniqueName: \"kubernetes.io/projected/6c26e207-3abb-40b7-9d56-b3f48d9d3ba4-kube-api-access-t745x\") pod \"console-operator-86d7c6cc77-qwjj5\" (UID: \"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4\") " pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:56.048170 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:56.048111 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:35:56.161220 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:56.161187 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-qwjj5"] Jun 02 21:35:56.165555 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:35:56.165530 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c26e207_3abb_40b7_9d56_b3f48d9d3ba4.slice/crio-e904692ec648861fc03ef7169d0a7265c2c0fca8061e28dc0ad28462cfb30b25 WatchSource:0}: Error finding container e904692ec648861fc03ef7169d0a7265c2c0fca8061e28dc0ad28462cfb30b25: Status 404 returned error can't find the container with id e904692ec648861fc03ef7169d0a7265c2c0fca8061e28dc0ad28462cfb30b25 Jun 02 21:35:56.441703 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:56.441661 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" event={"ID":"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4","Type":"ContainerStarted","Data":"e904692ec648861fc03ef7169d0a7265c2c0fca8061e28dc0ad28462cfb30b25"} Jun 02 21:35:56.471995 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:56.471973 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8v5xt_c1e9b389-999a-4410-8048-ff2e03df62dc/node-ca/0.log" Jun 02 21:35:57.500243 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:57.500211 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:35:57.500660 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:57.500257 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:57.500660 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:57.500294 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:35:57.500660 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:57.500337 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 21:35:57.500660 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:57.500393 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls podName:e6f2d34a-e816-4c95-9068-359e6d8f43ff nodeName:}" failed. No retries permitted until 2026-06-02 21:36:05.500378727 +0000 UTC m=+141.982963308 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-wdn2r" (UID: "e6f2d34a-e816-4c95-9068-359e6d8f43ff") : secret "cluster-monitoring-operator-tls" not found Jun 02 21:35:57.500660 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:57.500423 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle podName:37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce nodeName:}" failed. No retries permitted until 2026-06-02 21:36:05.500408374 +0000 UTC m=+141.982992965 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle") pod "router-default-86dd499c55-wbghk" (UID: "37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce") : configmap references non-existent config key: service-ca.crt Jun 02 21:35:57.500660 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:57.500425 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 21:35:57.500660 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:35:57.500495 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs podName:37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce nodeName:}" failed. No retries permitted until 2026-06-02 21:36:05.500479629 +0000 UTC m=+141.983064210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs") pod "router-default-86dd499c55-wbghk" (UID: "37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce") : secret "router-metrics-certs-default" not found Jun 02 21:35:59.449242 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:59.449214 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/0.log" Jun 02 21:35:59.449640 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:59.449257 2562 generic.go:358] "Generic (PLEG): container finished" podID="6c26e207-3abb-40b7-9d56-b3f48d9d3ba4" containerID="b24db84866fa213c07101d6b485cd1b77dc835d8ef6ac401afe5c14c33647fc7" exitCode=255 Jun 02 21:35:59.449640 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:59.449322 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" event={"ID":"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4","Type":"ContainerDied","Data":"b24db84866fa213c07101d6b485cd1b77dc835d8ef6ac401afe5c14c33647fc7"} Jun 02 21:35:59.449640 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:35:59.449535 2562 scope.go:117] "RemoveContainer" containerID="b24db84866fa213c07101d6b485cd1b77dc835d8ef6ac401afe5c14c33647fc7" Jun 02 21:36:00.452388 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:00.452352 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/1.log" Jun 02 21:36:00.452925 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:00.452831 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/0.log" Jun 02 21:36:00.452925 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:00.452889 2562 generic.go:358] "Generic (PLEG): container finished" podID="6c26e207-3abb-40b7-9d56-b3f48d9d3ba4" containerID="3204d7454db2946ad56f8590d70fd5ce7f68bd8f2e542beb1ae39cb17fb1360f" exitCode=255 Jun 02 21:36:00.453013 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:00.452941 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" event={"ID":"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4","Type":"ContainerDied","Data":"3204d7454db2946ad56f8590d70fd5ce7f68bd8f2e542beb1ae39cb17fb1360f"} Jun 02 21:36:00.453013 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:00.452978 2562 scope.go:117] "RemoveContainer" containerID="b24db84866fa213c07101d6b485cd1b77dc835d8ef6ac401afe5c14c33647fc7" Jun 02 21:36:00.453274 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:00.453245 2562 scope.go:117] "RemoveContainer" containerID="3204d7454db2946ad56f8590d70fd5ce7f68bd8f2e542beb1ae39cb17fb1360f" Jun 02 21:36:00.453461 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:00.453440 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-qwjj5_openshift-console-operator(6c26e207-3abb-40b7-9d56-b3f48d9d3ba4)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" podUID="6c26e207-3abb-40b7-9d56-b3f48d9d3ba4" Jun 02 21:36:01.456323 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:01.456294 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/1.log" Jun 02 21:36:01.456706 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:01.456624 2562 scope.go:117] "RemoveContainer" containerID="3204d7454db2946ad56f8590d70fd5ce7f68bd8f2e542beb1ae39cb17fb1360f" Jun 02 21:36:01.456823 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:01.456803 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-qwjj5_openshift-console-operator(6c26e207-3abb-40b7-9d56-b3f48d9d3ba4)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" podUID="6c26e207-3abb-40b7-9d56-b3f48d9d3ba4" Jun 02 21:36:04.909105 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:04.909067 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-w72h2"] Jun 02 21:36:04.913251 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:04.913233 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-w72h2" Jun 02 21:36:04.915517 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:04.915491 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 02 21:36:04.915629 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:04.915564 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 02 21:36:04.916460 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:04.916437 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-dfxln\"" Jun 02 21:36:04.919891 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:04.919850 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-w72h2"] Jun 02 21:36:05.050773 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:05.050741 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dx5c\" (UniqueName: \"kubernetes.io/projected/40456ffe-4d23-4e29-8a3d-845996885da8-kube-api-access-7dx5c\") pod \"migrator-96c6f749-w72h2\" (UID: \"40456ffe-4d23-4e29-8a3d-845996885da8\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-w72h2" Jun 02 21:36:05.151479 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:05.151453 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7dx5c\" (UniqueName: \"kubernetes.io/projected/40456ffe-4d23-4e29-8a3d-845996885da8-kube-api-access-7dx5c\") pod \"migrator-96c6f749-w72h2\" (UID: \"40456ffe-4d23-4e29-8a3d-845996885da8\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-w72h2" Jun 02 21:36:05.159631 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:05.159561 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dx5c\" (UniqueName: \"kubernetes.io/projected/40456ffe-4d23-4e29-8a3d-845996885da8-kube-api-access-7dx5c\") pod \"migrator-96c6f749-w72h2\" (UID: \"40456ffe-4d23-4e29-8a3d-845996885da8\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-w72h2" Jun 02 21:36:05.224576 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:05.224553 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-w72h2" Jun 02 21:36:05.341118 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:05.341088 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-w72h2"] Jun 02 21:36:05.343768 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:36:05.343740 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40456ffe_4d23_4e29_8a3d_845996885da8.slice/crio-84eecf9ee06a386ba82930409d4cdcc5aadf9f54fd8d790d8baaff8e35b0df00 WatchSource:0}: Error finding container 84eecf9ee06a386ba82930409d4cdcc5aadf9f54fd8d790d8baaff8e35b0df00: Status 404 returned error can't find the container with id 84eecf9ee06a386ba82930409d4cdcc5aadf9f54fd8d790d8baaff8e35b0df00 Jun 02 21:36:05.463367 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:05.463336 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-w72h2" event={"ID":"40456ffe-4d23-4e29-8a3d-845996885da8","Type":"ContainerStarted","Data":"84eecf9ee06a386ba82930409d4cdcc5aadf9f54fd8d790d8baaff8e35b0df00"} Jun 02 21:36:05.555151 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:05.555125 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:36:05.555261 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:05.555164 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:36:05.555261 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:05.555191 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:36:05.555367 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:05.555279 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 21:36:05.555367 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:05.555336 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle podName:37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce nodeName:}" failed. No retries permitted until 2026-06-02 21:36:21.555308967 +0000 UTC m=+158.037893553 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle") pod "router-default-86dd499c55-wbghk" (UID: "37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce") : configmap references non-existent config key: service-ca.crt Jun 02 21:36:05.555367 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:05.555362 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs podName:37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce nodeName:}" failed. No retries permitted until 2026-06-02 21:36:21.555353965 +0000 UTC m=+158.037938546 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs") pod "router-default-86dd499c55-wbghk" (UID: "37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce") : secret "router-metrics-certs-default" not found Jun 02 21:36:05.555486 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:05.555366 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 21:36:05.555486 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:05.555436 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls podName:e6f2d34a-e816-4c95-9068-359e6d8f43ff nodeName:}" failed. No retries permitted until 2026-06-02 21:36:21.5554147 +0000 UTC m=+158.037999282 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-wdn2r" (UID: "e6f2d34a-e816-4c95-9068-359e6d8f43ff") : secret "cluster-monitoring-operator-tls" not found Jun 02 21:36:06.048500 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:06.048468 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:36:06.048500 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:06.048506 2562 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:36:06.048899 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:06.048823 2562 scope.go:117] "RemoveContainer" containerID="3204d7454db2946ad56f8590d70fd5ce7f68bd8f2e542beb1ae39cb17fb1360f" Jun 02 21:36:06.049010 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:06.048993 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-qwjj5_openshift-console-operator(6c26e207-3abb-40b7-9d56-b3f48d9d3ba4)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" podUID="6c26e207-3abb-40b7-9d56-b3f48d9d3ba4" Jun 02 21:36:07.135943 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.135884 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lcgft"] Jun 02 21:36:07.138787 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.138769 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.141204 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.141181 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 21:36:07.141327 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.141188 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 21:36:07.141380 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.141350 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 21:36:07.142258 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.142238 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 21:36:07.142370 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.142263 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-stqmt\"" Jun 02 21:36:07.146547 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.146525 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lcgft"] Jun 02 21:36:07.268048 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.268027 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1f20a8a4-7f67-478e-a561-ae04f286726a-crio-socket\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.268154 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.268067 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f20a8a4-7f67-478e-a561-ae04f286726a-data-volume\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.268154 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.268095 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.268154 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.268121 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8l7p\" (UniqueName: \"kubernetes.io/projected/1f20a8a4-7f67-478e-a561-ae04f286726a-kube-api-access-s8l7p\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.268264 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.268176 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f20a8a4-7f67-478e-a561-ae04f286726a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.369108 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.369079 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1f20a8a4-7f67-478e-a561-ae04f286726a-crio-socket\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.369108 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.369115 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f20a8a4-7f67-478e-a561-ae04f286726a-data-volume\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.369253 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.369138 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.369253 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.369175 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s8l7p\" (UniqueName: \"kubernetes.io/projected/1f20a8a4-7f67-478e-a561-ae04f286726a-kube-api-access-s8l7p\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.369253 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.369199 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1f20a8a4-7f67-478e-a561-ae04f286726a-crio-socket\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.369253 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:07.369240 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 21:36:07.369385 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.369262 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f20a8a4-7f67-478e-a561-ae04f286726a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.369385 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:07.369299 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls podName:1f20a8a4-7f67-478e-a561-ae04f286726a nodeName:}" failed. No retries permitted until 2026-06-02 21:36:07.869278502 +0000 UTC m=+144.351863086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lcgft" (UID: "1f20a8a4-7f67-478e-a561-ae04f286726a") : secret "insights-runtime-extractor-tls" not found Jun 02 21:36:07.369470 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.369418 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f20a8a4-7f67-478e-a561-ae04f286726a-data-volume\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.369776 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.369759 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f20a8a4-7f67-478e-a561-ae04f286726a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.381356 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.381339 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8l7p\" (UniqueName: \"kubernetes.io/projected/1f20a8a4-7f67-478e-a561-ae04f286726a-kube-api-access-s8l7p\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.470630 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.470602 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-w72h2" event={"ID":"40456ffe-4d23-4e29-8a3d-845996885da8","Type":"ContainerStarted","Data":"fa10a7a3b231492832e91d645155a942d8be848d83dd3280887a15ac4c482e8b"} Jun 02 21:36:07.470726 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.470635 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-w72h2" event={"ID":"40456ffe-4d23-4e29-8a3d-845996885da8","Type":"ContainerStarted","Data":"e03e33c68473db08e66403a1f4d07760e9ffef42ca69f9b20f019b2285019966"} Jun 02 21:36:07.487575 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.487533 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-w72h2" podStartSLOduration=2.017293313 podStartE2EDuration="3.487521803s" podCreationTimestamp="2026-06-02 21:36:04 +0000 UTC" firstStartedPulling="2026-06-02 21:36:05.345531603 +0000 UTC m=+141.828116183" lastFinishedPulling="2026-06-02 21:36:06.815760092 +0000 UTC m=+143.298344673" observedRunningTime="2026-06-02 21:36:07.486278045 +0000 UTC m=+143.968862650" watchObservedRunningTime="2026-06-02 21:36:07.487521803 +0000 UTC m=+143.970106406" Jun 02 21:36:07.871940 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:07.871881 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:07.872032 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:07.871976 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 21:36:07.872032 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:07.872032 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls podName:1f20a8a4-7f67-478e-a561-ae04f286726a nodeName:}" failed. No retries permitted until 2026-06-02 21:36:08.872018055 +0000 UTC m=+145.354602635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lcgft" (UID: "1f20a8a4-7f67-478e-a561-ae04f286726a") : secret "insights-runtime-extractor-tls" not found Jun 02 21:36:08.878532 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:08.878488 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:08.878964 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:08.878638 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 21:36:08.878964 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:08.878709 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls podName:1f20a8a4-7f67-478e-a561-ae04f286726a nodeName:}" failed. No retries permitted until 2026-06-02 21:36:10.878692657 +0000 UTC m=+147.361277241 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lcgft" (UID: "1f20a8a4-7f67-478e-a561-ae04f286726a") : secret "insights-runtime-extractor-tls" not found Jun 02 21:36:10.893808 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:10.893764 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:10.894270 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:10.893929 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 21:36:10.894270 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:10.894000 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls podName:1f20a8a4-7f67-478e-a561-ae04f286726a nodeName:}" failed. No retries permitted until 2026-06-02 21:36:14.893983768 +0000 UTC m=+151.376568350 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lcgft" (UID: "1f20a8a4-7f67-478e-a561-ae04f286726a") : secret "insights-runtime-extractor-tls" not found Jun 02 21:36:14.921268 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:14.921233 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:14.924024 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:14.923995 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f20a8a4-7f67-478e-a561-ae04f286726a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lcgft\" (UID: \"1f20a8a4-7f67-478e-a561-ae04f286726a\") " pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:14.947524 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:14.947501 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lcgft" Jun 02 21:36:15.069619 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:15.069586 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lcgft"] Jun 02 21:36:15.072180 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:36:15.072148 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f20a8a4_7f67_478e_a561_ae04f286726a.slice/crio-59c57f2cc37333320b4ca3f3028732cfd5107ab1084a56bc6b45ea44c96b5c32 WatchSource:0}: Error finding container 59c57f2cc37333320b4ca3f3028732cfd5107ab1084a56bc6b45ea44c96b5c32: Status 404 returned error can't find the container with id 59c57f2cc37333320b4ca3f3028732cfd5107ab1084a56bc6b45ea44c96b5c32 Jun 02 21:36:15.489133 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:15.489093 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lcgft" event={"ID":"1f20a8a4-7f67-478e-a561-ae04f286726a","Type":"ContainerStarted","Data":"ed5b06f8573ccfae6a5f4ba8cbeb3872519a179ca4ff83888719d987ed5ff9a0"} Jun 02 21:36:15.489262 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:15.489138 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lcgft" event={"ID":"1f20a8a4-7f67-478e-a561-ae04f286726a","Type":"ContainerStarted","Data":"59c57f2cc37333320b4ca3f3028732cfd5107ab1084a56bc6b45ea44c96b5c32"} Jun 02 21:36:16.493314 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:16.493281 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lcgft" event={"ID":"1f20a8a4-7f67-478e-a561-ae04f286726a","Type":"ContainerStarted","Data":"5ca6754fbb5eab8886891d08c4dbc6f105bd1cb6ed9b83164b2f0b7c4c9cedc8"} Jun 02 21:36:17.496641 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:17.496608 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lcgft" event={"ID":"1f20a8a4-7f67-478e-a561-ae04f286726a","Type":"ContainerStarted","Data":"fe947e29cb8d3dbdd4e2c4343c228f07976d74edd3db2ef7c97f605d5a72fa35"} Jun 02 21:36:17.515314 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:17.515270 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lcgft" podStartSLOduration=8.226120945 podStartE2EDuration="10.515256334s" podCreationTimestamp="2026-06-02 21:36:07 +0000 UTC" firstStartedPulling="2026-06-02 21:36:15.125204413 +0000 UTC m=+151.607788994" lastFinishedPulling="2026-06-02 21:36:17.414339791 +0000 UTC m=+153.896924383" observedRunningTime="2026-06-02 21:36:17.514247192 +0000 UTC m=+153.996831807" watchObservedRunningTime="2026-06-02 21:36:17.515256334 +0000 UTC m=+153.997840936" Jun 02 21:36:18.983384 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:18.983346 2562 scope.go:117] "RemoveContainer" containerID="3204d7454db2946ad56f8590d70fd5ce7f68bd8f2e542beb1ae39cb17fb1360f" Jun 02 21:36:19.509061 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:19.509025 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:36:19.509474 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:19.509453 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/1.log" Jun 02 21:36:19.509587 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:19.509498 2562 generic.go:358] "Generic (PLEG): container finished" podID="6c26e207-3abb-40b7-9d56-b3f48d9d3ba4" containerID="99ce67c461c4e4c52c8c4b0d36c5100822d30a00aa19f86ac34ebecddddd7be0" exitCode=255 Jun 02 21:36:19.509587 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:19.509541 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" event={"ID":"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4","Type":"ContainerDied","Data":"99ce67c461c4e4c52c8c4b0d36c5100822d30a00aa19f86ac34ebecddddd7be0"} Jun 02 21:36:19.509587 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:19.509577 2562 scope.go:117] "RemoveContainer" containerID="3204d7454db2946ad56f8590d70fd5ce7f68bd8f2e542beb1ae39cb17fb1360f" Jun 02 21:36:19.509980 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:19.509958 2562 scope.go:117] "RemoveContainer" containerID="99ce67c461c4e4c52c8c4b0d36c5100822d30a00aa19f86ac34ebecddddd7be0" Jun 02 21:36:19.510185 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:19.510163 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-86d7c6cc77-qwjj5_openshift-console-operator(6c26e207-3abb-40b7-9d56-b3f48d9d3ba4)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" podUID="6c26e207-3abb-40b7-9d56-b3f48d9d3ba4" Jun 02 21:36:20.513878 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:20.513845 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:36:21.509005 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:21.508957 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bs2gf" podUID="d51d5fac-3da4-4fb8-a84d-6c9e748a91af" Jun 02 21:36:21.518271 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:21.518242 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bs2gf" Jun 02 21:36:21.542709 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:21.542679 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-vccz8" podUID="68e8e581-ba13-4f0c-8ffe-21bde0160fb7" Jun 02 21:36:21.567914 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:21.567883 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:36:21.567914 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:21.567925 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:36:21.568094 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:21.567954 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:36:21.568525 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:21.568497 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-service-ca-bundle\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:36:21.570611 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:21.570580 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce-metrics-certs\") pod \"router-default-86dd499c55-wbghk\" (UID: \"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce\") " pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:36:21.570611 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:21.570609 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6f2d34a-e816-4c95-9068-359e6d8f43ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-wdn2r\" (UID: \"e6f2d34a-e816-4c95-9068-359e6d8f43ff\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:36:21.573427 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:21.573407 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" Jun 02 21:36:21.580079 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:21.580055 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:36:21.705696 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:21.705670 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r"] Jun 02 21:36:21.709927 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:36:21.709893 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6f2d34a_e816_4c95_9068_359e6d8f43ff.slice/crio-b4403c9c572a828781751abc80f9e10dcdc07a37b7754091d7e8b64da3286a66 WatchSource:0}: Error finding container b4403c9c572a828781751abc80f9e10dcdc07a37b7754091d7e8b64da3286a66: Status 404 returned error can't find the container with id b4403c9c572a828781751abc80f9e10dcdc07a37b7754091d7e8b64da3286a66 Jun 02 21:36:21.722273 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:21.722247 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-86dd499c55-wbghk"] Jun 02 21:36:21.725100 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:36:21.725079 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37472cf6_e7ec_41ff_9ae7_3f54e7cfe4ce.slice/crio-5360eb9c7d7cb271de1189c2edc869e086e316bb508330b8c8dc28a65a964f32 WatchSource:0}: Error finding container 5360eb9c7d7cb271de1189c2edc869e086e316bb508330b8c8dc28a65a964f32: Status 404 returned error can't find the container with id 5360eb9c7d7cb271de1189c2edc869e086e316bb508330b8c8dc28a65a964f32 Jun 02 21:36:22.521841 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:22.521796 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" event={"ID":"e6f2d34a-e816-4c95-9068-359e6d8f43ff","Type":"ContainerStarted","Data":"b4403c9c572a828781751abc80f9e10dcdc07a37b7754091d7e8b64da3286a66"} Jun 02 21:36:22.523239 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:22.523215 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-86dd499c55-wbghk" event={"ID":"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce","Type":"ContainerStarted","Data":"d6fd1ec7518466b85cec455b074c736e9ed7deb07851d22357872424989480da"} Jun 02 21:36:22.523353 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:22.523243 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-86dd499c55-wbghk" event={"ID":"37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce","Type":"ContainerStarted","Data":"5360eb9c7d7cb271de1189c2edc869e086e316bb508330b8c8dc28a65a964f32"} Jun 02 21:36:22.541121 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:22.541077 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-86dd499c55-wbghk" podStartSLOduration=33.541063719 podStartE2EDuration="33.541063719s" podCreationTimestamp="2026-06-02 21:35:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:36:22.539892528 +0000 UTC m=+159.022477125" watchObservedRunningTime="2026-06-02 21:36:22.541063719 +0000 UTC m=+159.023648323" Jun 02 21:36:22.581139 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:22.581110 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:36:22.583368 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:22.583348 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:36:23.002364 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:23.002336 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-p58r9" podUID="b4c54dfc-ecd1-466b-90db-eca67b4780c5" Jun 02 21:36:23.526134 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:23.526096 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:36:23.527443 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:23.527416 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-86dd499c55-wbghk" Jun 02 21:36:24.529652 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:24.529602 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" event={"ID":"e6f2d34a-e816-4c95-9068-359e6d8f43ff","Type":"ContainerStarted","Data":"d873281df2d6d3c1d7ee4159b3ed3f64697764f3304d256dffa6f9c690691fdf"} Jun 02 21:36:24.546424 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:24.546372 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-wdn2r" podStartSLOduration=32.796233752 podStartE2EDuration="35.546354075s" podCreationTimestamp="2026-06-02 21:35:49 +0000 UTC" firstStartedPulling="2026-06-02 21:36:21.711665947 +0000 UTC m=+158.194250528" lastFinishedPulling="2026-06-02 21:36:24.461786268 +0000 UTC m=+160.944370851" observedRunningTime="2026-06-02 21:36:24.545388986 +0000 UTC m=+161.027973582" watchObservedRunningTime="2026-06-02 21:36:24.546354075 +0000 UTC m=+161.028938680" Jun 02 21:36:26.049055 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.049025 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:36:26.049531 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.049103 2562 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:36:26.049531 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.049434 2562 scope.go:117] "RemoveContainer" containerID="99ce67c461c4e4c52c8c4b0d36c5100822d30a00aa19f86ac34ebecddddd7be0" Jun 02 21:36:26.049673 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:26.049641 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-86d7c6cc77-qwjj5_openshift-console-operator(6c26e207-3abb-40b7-9d56-b3f48d9d3ba4)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" podUID="6c26e207-3abb-40b7-9d56-b3f48d9d3ba4" Jun 02 21:36:26.300559 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.300488 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c6977d9cd-8d7tq"] Jun 02 21:36:26.303401 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.303383 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.306099 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.306074 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-x4k7h\"" Jun 02 21:36:26.306230 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.306109 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 21:36:26.306230 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.306116 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 21:36:26.306230 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.306141 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 21:36:26.311802 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.311762 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 21:36:26.316236 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.316214 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c6977d9cd-8d7tq"] Jun 02 21:36:26.403487 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.403462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2987d45b-7b68-4d07-a82f-01202b7fc38e-ca-trust-extracted\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.403592 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.403494 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2987d45b-7b68-4d07-a82f-01202b7fc38e-bound-sa-token\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.403592 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.403516 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2987d45b-7b68-4d07-a82f-01202b7fc38e-image-registry-private-configuration\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.403592 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.403536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2987d45b-7b68-4d07-a82f-01202b7fc38e-registry-certificates\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.403703 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.403613 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:36:26.403703 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.403640 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2987d45b-7b68-4d07-a82f-01202b7fc38e-registry-tls\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.403703 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.403661 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:36:26.403703 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.403687 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2987d45b-7b68-4d07-a82f-01202b7fc38e-trusted-ca\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.403833 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.403724 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2987d45b-7b68-4d07-a82f-01202b7fc38e-installation-pull-secrets\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.403833 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.403745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c55n9\" (UniqueName: \"kubernetes.io/projected/2987d45b-7b68-4d07-a82f-01202b7fc38e-kube-api-access-c55n9\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.405947 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.405925 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d51d5fac-3da4-4fb8-a84d-6c9e748a91af-metrics-tls\") pod \"dns-default-bs2gf\" (UID: \"d51d5fac-3da4-4fb8-a84d-6c9e748a91af\") " pod="openshift-dns/dns-default-bs2gf" Jun 02 21:36:26.406056 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.406040 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8e581-ba13-4f0c-8ffe-21bde0160fb7-cert\") pod \"ingress-canary-vccz8\" (UID: \"68e8e581-ba13-4f0c-8ffe-21bde0160fb7\") " pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:36:26.504203 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.504172 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2987d45b-7b68-4d07-a82f-01202b7fc38e-bound-sa-token\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.504310 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.504216 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2987d45b-7b68-4d07-a82f-01202b7fc38e-image-registry-private-configuration\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.504310 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.504244 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2987d45b-7b68-4d07-a82f-01202b7fc38e-registry-certificates\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.504310 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.504300 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2987d45b-7b68-4d07-a82f-01202b7fc38e-registry-tls\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.504480 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.504344 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2987d45b-7b68-4d07-a82f-01202b7fc38e-trusted-ca\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.504480 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.504374 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2987d45b-7b68-4d07-a82f-01202b7fc38e-installation-pull-secrets\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.504480 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.504402 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c55n9\" (UniqueName: \"kubernetes.io/projected/2987d45b-7b68-4d07-a82f-01202b7fc38e-kube-api-access-c55n9\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.504480 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.504434 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2987d45b-7b68-4d07-a82f-01202b7fc38e-ca-trust-extracted\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.504949 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.504904 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2987d45b-7b68-4d07-a82f-01202b7fc38e-ca-trust-extracted\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.505404 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.505376 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2987d45b-7b68-4d07-a82f-01202b7fc38e-registry-certificates\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.505626 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.505601 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2987d45b-7b68-4d07-a82f-01202b7fc38e-trusted-ca\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.506580 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.506557 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2987d45b-7b68-4d07-a82f-01202b7fc38e-image-registry-private-configuration\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.506814 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.506794 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2987d45b-7b68-4d07-a82f-01202b7fc38e-installation-pull-secrets\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.507036 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.507019 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2987d45b-7b68-4d07-a82f-01202b7fc38e-registry-tls\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.514512 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.514483 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2987d45b-7b68-4d07-a82f-01202b7fc38e-bound-sa-token\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.514861 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.514839 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c55n9\" (UniqueName: \"kubernetes.io/projected/2987d45b-7b68-4d07-a82f-01202b7fc38e-kube-api-access-c55n9\") pod \"image-registry-6c6977d9cd-8d7tq\" (UID: \"2987d45b-7b68-4d07-a82f-01202b7fc38e\") " pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.535247 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.535230 2562 scope.go:117] "RemoveContainer" containerID="99ce67c461c4e4c52c8c4b0d36c5100822d30a00aa19f86ac34ebecddddd7be0" Jun 02 21:36:26.535396 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:26.535381 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-86d7c6cc77-qwjj5_openshift-console-operator(6c26e207-3abb-40b7-9d56-b3f48d9d3ba4)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" podUID="6c26e207-3abb-40b7-9d56-b3f48d9d3ba4" Jun 02 21:36:26.613085 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.613027 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:26.620882 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.620846 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gs96v\"" Jun 02 21:36:26.629137 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.629114 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bs2gf" Jun 02 21:36:26.749888 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.748638 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c6977d9cd-8d7tq"] Jun 02 21:36:26.753263 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:36:26.753236 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2987d45b_7b68_4d07_a82f_01202b7fc38e.slice/crio-1020a5d6a72965642479972e07b62546b75a3849676fcf9ceb1c692bb4c16c4a WatchSource:0}: Error finding container 1020a5d6a72965642479972e07b62546b75a3849676fcf9ceb1c692bb4c16c4a: Status 404 returned error can't find the container with id 1020a5d6a72965642479972e07b62546b75a3849676fcf9ceb1c692bb4c16c4a Jun 02 21:36:26.760288 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:26.760148 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bs2gf"] Jun 02 21:36:26.765219 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:36:26.765195 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd51d5fac_3da4_4fb8_a84d_6c9e748a91af.slice/crio-153e83af6e53e01a70890a70fa65c3b0746ebd3182b1873e4770ec413d5ebf42 WatchSource:0}: Error finding container 153e83af6e53e01a70890a70fa65c3b0746ebd3182b1873e4770ec413d5ebf42: Status 404 returned error can't find the container with id 153e83af6e53e01a70890a70fa65c3b0746ebd3182b1873e4770ec413d5ebf42 Jun 02 21:36:27.538495 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:27.538456 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bs2gf" event={"ID":"d51d5fac-3da4-4fb8-a84d-6c9e748a91af","Type":"ContainerStarted","Data":"153e83af6e53e01a70890a70fa65c3b0746ebd3182b1873e4770ec413d5ebf42"} Jun 02 21:36:27.539930 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:27.539900 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" event={"ID":"2987d45b-7b68-4d07-a82f-01202b7fc38e","Type":"ContainerStarted","Data":"3677a6e82f4a9bcbfbd2a5d639d9c32f2ab438fa0994cd0e94393e304458a024"} Jun 02 21:36:27.540058 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:27.539942 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" event={"ID":"2987d45b-7b68-4d07-a82f-01202b7fc38e","Type":"ContainerStarted","Data":"1020a5d6a72965642479972e07b62546b75a3849676fcf9ceb1c692bb4c16c4a"} Jun 02 21:36:27.540118 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:27.540053 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:27.560901 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:27.560847 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" podStartSLOduration=1.56083447 podStartE2EDuration="1.56083447s" podCreationTimestamp="2026-06-02 21:36:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:36:27.559282367 +0000 UTC m=+164.041866973" watchObservedRunningTime="2026-06-02 21:36:27.56083447 +0000 UTC m=+164.043419073" Jun 02 21:36:28.546461 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:28.546422 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bs2gf" event={"ID":"d51d5fac-3da4-4fb8-a84d-6c9e748a91af","Type":"ContainerStarted","Data":"2f210e6f57a3470ad17f18277a5f895fe6090bce95ce8d08c8eca759ccdde02b"} Jun 02 21:36:28.546461 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:28.546468 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bs2gf" event={"ID":"d51d5fac-3da4-4fb8-a84d-6c9e748a91af","Type":"ContainerStarted","Data":"76bca27e010197d2761425a85290c7688a19869a98bce57e5d9f57eefff53aca"} Jun 02 21:36:28.546934 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:28.546659 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bs2gf" Jun 02 21:36:28.563167 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:28.563113 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-bs2gf" podStartSLOduration=129.40275273 podStartE2EDuration="2m10.563096968s" podCreationTimestamp="2026-06-02 21:34:18 +0000 UTC" firstStartedPulling="2026-06-02 21:36:26.766915033 +0000 UTC m=+163.249499613" lastFinishedPulling="2026-06-02 21:36:27.927259257 +0000 UTC m=+164.409843851" observedRunningTime="2026-06-02 21:36:28.563020715 +0000 UTC m=+165.045605318" watchObservedRunningTime="2026-06-02 21:36:28.563096968 +0000 UTC m=+165.045681571" Jun 02 21:36:32.983181 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:32.983140 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:36:32.985847 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:32.985825 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2lz6l\"" Jun 02 21:36:32.993929 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:32.993913 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vccz8" Jun 02 21:36:33.119241 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:33.119212 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vccz8"] Jun 02 21:36:33.122576 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:36:33.122551 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e8e581_ba13_4f0c_8ffe_21bde0160fb7.slice/crio-ff172668473d0e24c144b44fbd050b3e306ca984b3b3aa3adf7f0112c457f38f WatchSource:0}: Error finding container ff172668473d0e24c144b44fbd050b3e306ca984b3b3aa3adf7f0112c457f38f: Status 404 returned error can't find the container with id ff172668473d0e24c144b44fbd050b3e306ca984b3b3aa3adf7f0112c457f38f Jun 02 21:36:33.562091 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:33.562053 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vccz8" event={"ID":"68e8e581-ba13-4f0c-8ffe-21bde0160fb7","Type":"ContainerStarted","Data":"ff172668473d0e24c144b44fbd050b3e306ca984b3b3aa3adf7f0112c457f38f"} Jun 02 21:36:33.985260 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:33.985203 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:36:35.344762 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.344729 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-xh8k8"] Jun 02 21:36:35.346656 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.346633 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:35.350082 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.350062 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bngfj\"" Jun 02 21:36:35.350217 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.350201 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 02 21:36:35.351150 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.351129 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 02 21:36:35.351230 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.351163 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 21:36:35.356296 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.356277 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9pt47"] Jun 02 21:36:35.358116 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.358095 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.359686 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.359665 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b"] Jun 02 21:36:35.360734 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.360714 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 21:36:35.361090 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.361070 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 21:36:35.361457 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.361337 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 21:36:35.361830 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.361813 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-xh8k8"] Jun 02 21:36:35.361936 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.361915 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-htktv\"" Jun 02 21:36:35.362155 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.361952 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.364306 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.364279 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-smr4c\"" Jun 02 21:36:35.364397 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.364378 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 02 21:36:35.364830 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.364714 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 02 21:36:35.365254 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.365113 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 02 21:36:35.379468 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.379446 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b"] Jun 02 21:36:35.468341 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl9c4\" (UniqueName: \"kubernetes.io/projected/b79a288e-785b-4ac6-b08d-fdc03727a420-kube-api-access-wl9c4\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.468341 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468339 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-root\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.468496 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468359 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b79a288e-785b-4ac6-b08d-fdc03727a420-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.468496 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468379 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/290ed1fe-5e18-451c-9825-f43cf21909e4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:35.468496 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468421 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh9p6\" (UniqueName: \"kubernetes.io/projected/290ed1fe-5e18-451c-9825-f43cf21909e4-kube-api-access-fh9p6\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:35.468496 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468455 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-metrics-client-ca\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.468615 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468503 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.468615 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.468615 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468550 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/290ed1fe-5e18-451c-9825-f43cf21909e4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:35.468615 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468567 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b79a288e-785b-4ac6-b08d-fdc03727a420-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.468615 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468586 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b79a288e-785b-4ac6-b08d-fdc03727a420-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.468759 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468640 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-tls\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.468759 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468679 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5d26\" (UniqueName: \"kubernetes.io/projected/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-kube-api-access-t5d26\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.468759 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468710 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/290ed1fe-5e18-451c-9825-f43cf21909e4-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:35.468759 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468727 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-wtmp\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.468759 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468743 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-sys\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.468928 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468766 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-textfile\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.468928 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468783 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b79a288e-785b-4ac6-b08d-fdc03727a420-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.468928 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.468800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b79a288e-785b-4ac6-b08d-fdc03727a420-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.568779 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.568753 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vccz8" event={"ID":"68e8e581-ba13-4f0c-8ffe-21bde0160fb7","Type":"ContainerStarted","Data":"d57687e07432572d28247c8bb2713e4e9f99e7381c5c7501c8a76eb23ce6762a"} Jun 02 21:36:35.569065 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569045 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/290ed1fe-5e18-451c-9825-f43cf21909e4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:35.569126 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569081 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fh9p6\" (UniqueName: \"kubernetes.io/projected/290ed1fe-5e18-451c-9825-f43cf21909e4-kube-api-access-fh9p6\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:35.569126 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569108 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-metrics-client-ca\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.569209 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569140 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.569209 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569168 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.569209 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569195 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/290ed1fe-5e18-451c-9825-f43cf21909e4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:35.569385 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569221 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b79a288e-785b-4ac6-b08d-fdc03727a420-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.569385 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569247 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b79a288e-785b-4ac6-b08d-fdc03727a420-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.569385 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569283 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-tls\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.569385 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569314 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t5d26\" (UniqueName: \"kubernetes.io/projected/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-kube-api-access-t5d26\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.569385 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:35.569342 2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 02 21:36:35.569385 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569346 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/290ed1fe-5e18-451c-9825-f43cf21909e4-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:35.569385 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569372 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-wtmp\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.569707 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569398 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-sys\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.569707 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:35.569411 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/290ed1fe-5e18-451c-9825-f43cf21909e4-openshift-state-metrics-tls podName:290ed1fe-5e18-451c-9825-f43cf21909e4 nodeName:}" failed. No retries permitted until 2026-06-02 21:36:36.069391658 +0000 UTC m=+172.551976244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/290ed1fe-5e18-451c-9825-f43cf21909e4-openshift-state-metrics-tls") pod "openshift-state-metrics-85658796db-xh8k8" (UID: "290ed1fe-5e18-451c-9825-f43cf21909e4") : secret "openshift-state-metrics-tls" not found Jun 02 21:36:35.569707 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569443 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-sys\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.569707 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569453 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-textfile\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.569707 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:35.569460 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 02 21:36:35.569707 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569485 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b79a288e-785b-4ac6-b08d-fdc03727a420-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.569707 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:35.569522 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-tls podName:e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec nodeName:}" failed. No retries permitted until 2026-06-02 21:36:36.069498966 +0000 UTC m=+172.552083557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-tls") pod "node-exporter-9pt47" (UID: "e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec") : secret "node-exporter-tls" not found Jun 02 21:36:35.569707 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569553 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b79a288e-785b-4ac6-b08d-fdc03727a420-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.569707 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569598 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9c4\" (UniqueName: \"kubernetes.io/projected/b79a288e-785b-4ac6-b08d-fdc03727a420-kube-api-access-wl9c4\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.569707 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569627 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-root\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.569707 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.569659 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b79a288e-785b-4ac6-b08d-fdc03727a420-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.570256 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.570013 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.570256 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.570152 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-wtmp\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.570256 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.570237 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-metrics-client-ca\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.570397 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.570262 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b79a288e-785b-4ac6-b08d-fdc03727a420-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.570397 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.570279 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b79a288e-785b-4ac6-b08d-fdc03727a420-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.570397 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.570325 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-root\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.570547 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.570530 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-textfile\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.570788 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.570760 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/290ed1fe-5e18-451c-9825-f43cf21909e4-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:35.570963 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.570941 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b79a288e-785b-4ac6-b08d-fdc03727a420-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.571997 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.571966 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/290ed1fe-5e18-451c-9825-f43cf21909e4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:35.572299 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.572279 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.573126 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.573103 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b79a288e-785b-4ac6-b08d-fdc03727a420-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.573206 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.573172 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b79a288e-785b-4ac6-b08d-fdc03727a420-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.586791 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.586764 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh9p6\" (UniqueName: \"kubernetes.io/projected/290ed1fe-5e18-451c-9825-f43cf21909e4-kube-api-access-fh9p6\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:35.587052 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.587034 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl9c4\" (UniqueName: \"kubernetes.io/projected/b79a288e-785b-4ac6-b08d-fdc03727a420-kube-api-access-wl9c4\") pod \"kube-state-metrics-8469c486bb-wkw9b\" (UID: \"b79a288e-785b-4ac6-b08d-fdc03727a420\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.587538 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.587518 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5d26\" (UniqueName: \"kubernetes.io/projected/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-kube-api-access-t5d26\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:35.593210 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.593171 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vccz8" podStartSLOduration=136.138052951 podStartE2EDuration="2m17.59315965s" podCreationTimestamp="2026-06-02 21:34:18 +0000 UTC" firstStartedPulling="2026-06-02 21:36:33.124885136 +0000 UTC m=+169.607469722" lastFinishedPulling="2026-06-02 21:36:34.579991836 +0000 UTC m=+171.062576421" observedRunningTime="2026-06-02 21:36:35.592555426 +0000 UTC m=+172.075140025" watchObservedRunningTime="2026-06-02 21:36:35.59315965 +0000 UTC m=+172.075744320" Jun 02 21:36:35.673109 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.673057 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" Jun 02 21:36:35.797069 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:35.797047 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b"] Jun 02 21:36:35.799408 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:36:35.799381 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb79a288e_785b_4ac6_b08d_fdc03727a420.slice/crio-aa5306bd92de35d10122997d638509a26cb51e175cc3b24eb86bff17dcc74838 WatchSource:0}: Error finding container aa5306bd92de35d10122997d638509a26cb51e175cc3b24eb86bff17dcc74838: Status 404 returned error can't find the container with id aa5306bd92de35d10122997d638509a26cb51e175cc3b24eb86bff17dcc74838 Jun 02 21:36:36.072511 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.072485 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/290ed1fe-5e18-451c-9825-f43cf21909e4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:36.072650 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.072527 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-tls\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:36.074862 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.074834 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec-node-exporter-tls\") pod \"node-exporter-9pt47\" (UID: \"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec\") " pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:36.074976 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.074925 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/290ed1fe-5e18-451c-9825-f43cf21909e4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-xh8k8\" (UID: \"290ed1fe-5e18-451c-9825-f43cf21909e4\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:36.255955 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.255928 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" Jun 02 21:36:36.268633 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.268609 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9pt47" Jun 02 21:36:36.405383 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.405351 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-xh8k8"] Jun 02 21:36:36.409330 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:36:36.409302 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod290ed1fe_5e18_451c_9825_f43cf21909e4.slice/crio-19e4bd833421df5df13cf8c491b10376e99e31c53eddc23201bc5534460da178 WatchSource:0}: Error finding container 19e4bd833421df5df13cf8c491b10376e99e31c53eddc23201bc5534460da178: Status 404 returned error can't find the container with id 19e4bd833421df5df13cf8c491b10376e99e31c53eddc23201bc5534460da178 Jun 02 21:36:36.572979 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.572923 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" event={"ID":"b79a288e-785b-4ac6-b08d-fdc03727a420","Type":"ContainerStarted","Data":"aa5306bd92de35d10122997d638509a26cb51e175cc3b24eb86bff17dcc74838"} Jun 02 21:36:36.574260 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.574236 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9pt47" event={"ID":"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec","Type":"ContainerStarted","Data":"cdfc60ce87be76c6b6aecc43c702b264f140da2a88b61b9324881f1430509469"} Jun 02 21:36:36.576122 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.576087 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" event={"ID":"290ed1fe-5e18-451c-9825-f43cf21909e4","Type":"ContainerStarted","Data":"30608e2c14191387374271f4b300f77991f5c09432f3470b7de23225728e5202"} Jun 02 21:36:36.576223 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.576128 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" event={"ID":"290ed1fe-5e18-451c-9825-f43cf21909e4","Type":"ContainerStarted","Data":"9913bfdc49ec0f6948db7e42cb2134b58e57beba3975c5f3c3d1cdc03a0940e2"} Jun 02 21:36:36.576223 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.576143 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" event={"ID":"290ed1fe-5e18-451c-9825-f43cf21909e4","Type":"ContainerStarted","Data":"19e4bd833421df5df13cf8c491b10376e99e31c53eddc23201bc5534460da178"} Jun 02 21:36:36.984171 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:36.984102 2562 scope.go:117] "RemoveContainer" containerID="99ce67c461c4e4c52c8c4b0d36c5100822d30a00aa19f86ac34ebecddddd7be0" Jun 02 21:36:36.984373 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:36:36.984355 2562 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-86d7c6cc77-qwjj5_openshift-console-operator(6c26e207-3abb-40b7-9d56-b3f48d9d3ba4)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" podUID="6c26e207-3abb-40b7-9d56-b3f48d9d3ba4" Jun 02 21:36:37.580752 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:37.580614 2562 generic.go:358] "Generic (PLEG): container finished" podID="e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec" containerID="23833cd5de523c746a9368963a998d6d74e5d8c32ecd94058f406f17be1cd7c0" exitCode=0 Jun 02 21:36:37.580752 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:37.580698 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9pt47" event={"ID":"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec","Type":"ContainerDied","Data":"23833cd5de523c746a9368963a998d6d74e5d8c32ecd94058f406f17be1cd7c0"} Jun 02 21:36:37.583316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:37.583252 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" event={"ID":"b79a288e-785b-4ac6-b08d-fdc03727a420","Type":"ContainerStarted","Data":"c9fe520f136e9f392cf088d15740123c680ae7bbab95b010f93c19474247ad0a"} Jun 02 21:36:37.583316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:37.583285 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" event={"ID":"b79a288e-785b-4ac6-b08d-fdc03727a420","Type":"ContainerStarted","Data":"be0465f88226d4e990ab372db0d47371b6f4238756995f5a2a53afde15e9a0b2"} Jun 02 21:36:37.583316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:37.583298 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" event={"ID":"b79a288e-785b-4ac6-b08d-fdc03727a420","Type":"ContainerStarted","Data":"28c197f5ffdf092222f851ff8d75dc192e80f81c58b746cb3f3992ac1cf1bf06"} Jun 02 21:36:37.626472 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:37.626425 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-wkw9b" podStartSLOduration=1.421274205 podStartE2EDuration="2.626412807s" podCreationTimestamp="2026-06-02 21:36:35 +0000 UTC" firstStartedPulling="2026-06-02 21:36:35.801288227 +0000 UTC m=+172.283872808" lastFinishedPulling="2026-06-02 21:36:37.006426825 +0000 UTC m=+173.489011410" observedRunningTime="2026-06-02 21:36:37.626056214 +0000 UTC m=+174.108640818" watchObservedRunningTime="2026-06-02 21:36:37.626412807 +0000 UTC m=+174.108997409" Jun 02 21:36:38.424970 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.424939 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr"] Jun 02 21:36:38.428097 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.428080 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.430843 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.430809 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 02 21:36:38.430996 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.430856 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d0oj3g8rh0r3o\"" Jun 02 21:36:38.431172 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.431141 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 02 21:36:38.431231 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.431211 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 02 21:36:38.431305 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.431288 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 02 21:36:38.431425 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.431410 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-gh6cf\"" Jun 02 21:36:38.431480 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.431467 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 02 21:36:38.439961 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.439940 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr"] Jun 02 21:36:38.551901 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.551862 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bs2gf" Jun 02 21:36:38.588928 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.588897 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9pt47" event={"ID":"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec","Type":"ContainerStarted","Data":"bf9b9066d51edea8802c7af4c264e6d76c17e216642425bb91f7a51cb49b1b1e"} Jun 02 21:36:38.589329 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.588934 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9pt47" event={"ID":"e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec","Type":"ContainerStarted","Data":"bef9ec4fcebbde2c780c6ddc081d878e579d9c51690d3d0a2c13091a4a0e45f2"} Jun 02 21:36:38.591538 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.591514 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" event={"ID":"290ed1fe-5e18-451c-9825-f43cf21909e4","Type":"ContainerStarted","Data":"b1f00d6a4473742ce5158ed6d1c6840bd9f29d95944220f51bab21bc3e933244"} Jun 02 21:36:38.591810 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.591792 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-tls\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.591910 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.591843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.591910 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.591896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.592017 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.591929 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dl8h\" (UniqueName: \"kubernetes.io/projected/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-kube-api-access-4dl8h\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.592017 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.591972 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-metrics-client-ca\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.592178 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.592157 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.592234 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.592208 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-grpc-tls\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.592287 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.592260 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.612674 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.612631 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9pt47" podStartSLOduration=2.57855528 podStartE2EDuration="3.612618379s" podCreationTimestamp="2026-06-02 21:36:35 +0000 UTC" firstStartedPulling="2026-06-02 21:36:36.279322112 +0000 UTC m=+172.761906701" lastFinishedPulling="2026-06-02 21:36:37.313385206 +0000 UTC m=+173.795969800" observedRunningTime="2026-06-02 21:36:38.611928429 +0000 UTC m=+175.094513025" watchObservedRunningTime="2026-06-02 21:36:38.612618379 +0000 UTC m=+175.095203040" Jun 02 21:36:38.628952 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.628913 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-xh8k8" podStartSLOduration=2.350962925 podStartE2EDuration="3.628901255s" podCreationTimestamp="2026-06-02 21:36:35 +0000 UTC" firstStartedPulling="2026-06-02 21:36:36.526536697 +0000 UTC m=+173.009121285" lastFinishedPulling="2026-06-02 21:36:37.804475035 +0000 UTC m=+174.287059615" observedRunningTime="2026-06-02 21:36:38.628328425 +0000 UTC m=+175.110913027" watchObservedRunningTime="2026-06-02 21:36:38.628901255 +0000 UTC m=+175.111485857" Jun 02 21:36:38.693108 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.693085 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4dl8h\" (UniqueName: \"kubernetes.io/projected/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-kube-api-access-4dl8h\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.693191 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.693148 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-metrics-client-ca\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.693779 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.693417 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.693779 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.693619 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-grpc-tls\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.693779 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.693678 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.694088 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.693841 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-tls\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.694088 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.693895 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.694088 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.693974 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.694088 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.694055 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-metrics-client-ca\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.696399 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.696377 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-grpc-tls\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.696484 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.696418 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.696656 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.696640 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.697038 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.697013 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.697141 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.697125 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-tls\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.697199 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.697164 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.702068 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.702036 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dl8h\" (UniqueName: \"kubernetes.io/projected/8e77aa53-8d9e-4724-89ac-5e6604cd48a5-kube-api-access-4dl8h\") pod \"thanos-querier-66bd8fbcc5-rqjtr\" (UID: \"8e77aa53-8d9e-4724-89ac-5e6604cd48a5\") " pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.737060 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.737035 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:38.866388 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:38.866358 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr"] Jun 02 21:36:38.869228 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:36:38.869204 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e77aa53_8d9e_4724_89ac_5e6604cd48a5.slice/crio-85fb9e06dcdba4f3df86f97acc5f5fec1b3a334d5a319fb6a372161c2372a2f3 WatchSource:0}: Error finding container 85fb9e06dcdba4f3df86f97acc5f5fec1b3a334d5a319fb6a372161c2372a2f3: Status 404 returned error can't find the container with id 85fb9e06dcdba4f3df86f97acc5f5fec1b3a334d5a319fb6a372161c2372a2f3 Jun 02 21:36:39.595681 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:39.595644 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" event={"ID":"8e77aa53-8d9e-4724-89ac-5e6604cd48a5","Type":"ContainerStarted","Data":"85fb9e06dcdba4f3df86f97acc5f5fec1b3a334d5a319fb6a372161c2372a2f3"} Jun 02 21:36:40.112506 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:40.112467 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj"] Jun 02 21:36:40.115553 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:40.115534 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj" Jun 02 21:36:40.118798 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:40.118771 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-fswhb\"" Jun 02 21:36:40.119080 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:40.119047 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 02 21:36:40.126378 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:40.126220 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj"] Jun 02 21:36:40.307936 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:40.307905 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6a49d7a6-3d14-4f12-8d83-8261bce45960-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-472dj\" (UID: \"6a49d7a6-3d14-4f12-8d83-8261bce45960\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj" Jun 02 21:36:40.408798 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:40.408732 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6a49d7a6-3d14-4f12-8d83-8261bce45960-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-472dj\" (UID: \"6a49d7a6-3d14-4f12-8d83-8261bce45960\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj" Jun 02 21:36:40.411308 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:40.411277 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6a49d7a6-3d14-4f12-8d83-8261bce45960-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-472dj\" (UID: \"6a49d7a6-3d14-4f12-8d83-8261bce45960\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj" Jun 02 21:36:40.428277 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:40.428258 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj" Jun 02 21:36:40.573982 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:40.573956 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj"] Jun 02 21:36:40.576710 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:36:40.576680 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a49d7a6_3d14_4f12_8d83_8261bce45960.slice/crio-937309140cc05ee055d095abffff4a6aee23219138e6ccd422276e26a8f4f054 WatchSource:0}: Error finding container 937309140cc05ee055d095abffff4a6aee23219138e6ccd422276e26a8f4f054: Status 404 returned error can't find the container with id 937309140cc05ee055d095abffff4a6aee23219138e6ccd422276e26a8f4f054 Jun 02 21:36:40.603650 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:40.603624 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj" event={"ID":"6a49d7a6-3d14-4f12-8d83-8261bce45960","Type":"ContainerStarted","Data":"937309140cc05ee055d095abffff4a6aee23219138e6ccd422276e26a8f4f054"} Jun 02 21:36:41.571810 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.571736 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 21:36:41.576103 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.576081 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.578524 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.578500 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nqp8j\"" Jun 02 21:36:41.580914 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.580892 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 02 21:36:41.581460 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.581440 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 02 21:36:41.581600 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.581586 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 02 21:36:41.581731 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.581713 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 02 21:36:41.581808 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.581789 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 02 21:36:41.582408 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.582386 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 02 21:36:41.583075 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.582758 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 02 21:36:41.583075 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.582773 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 02 21:36:41.583075 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.582824 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 02 21:36:41.583075 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.582840 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-an31mtppglrsi\"" Jun 02 21:36:41.583075 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.583039 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 02 21:36:41.583299 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.583194 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 02 21:36:41.585548 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.585529 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 02 21:36:41.588635 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.588587 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 02 21:36:41.589309 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.589286 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 21:36:41.612576 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.612544 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" event={"ID":"8e77aa53-8d9e-4724-89ac-5e6604cd48a5","Type":"ContainerStarted","Data":"3dbbd5f84a5d7451fd67ad82e03f7f751278d0240e396994bc73e2d56d3f2128"} Jun 02 21:36:41.612576 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.612575 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" event={"ID":"8e77aa53-8d9e-4724-89ac-5e6604cd48a5","Type":"ContainerStarted","Data":"8a069042fcf3ebbda9f340dbda31126ad5e8e383c9fa1b181d6b3233504807e9"} Jun 02 21:36:41.612902 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.612589 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" event={"ID":"8e77aa53-8d9e-4724-89ac-5e6604cd48a5","Type":"ContainerStarted","Data":"019d86f6ce320f61a07dacac412aa65fa7b1777fc880973e5726361b2fe5a1a4"} Jun 02 21:36:41.718350 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718325 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718491 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718360 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718491 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718380 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718491 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718409 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718636 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718636 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718636 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718537 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-config\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718636 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/69eab127-a507-40f7-b40a-1ab4967dc8bc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718804 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718635 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718804 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718804 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718697 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718804 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718761 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718804 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718786 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-web-config\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.718804 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718802 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/69eab127-a507-40f7-b40a-1ab4967dc8bc-config-out\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.719035 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718820 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.719035 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718901 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.719035 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718925 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9psdk\" (UniqueName: \"kubernetes.io/projected/69eab127-a507-40f7-b40a-1ab4967dc8bc-kube-api-access-9psdk\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.719035 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.718959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820299 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820271 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820405 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820308 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820405 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820345 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820405 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820378 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820573 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820408 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820573 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820436 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-config\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820573 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820463 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/69eab127-a507-40f7-b40a-1ab4967dc8bc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820573 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820561 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820753 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820599 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820753 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820630 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820753 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820682 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820753 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820715 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-web-config\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.820753 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820747 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/69eab127-a507-40f7-b40a-1ab4967dc8bc-config-out\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.821032 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820777 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.821032 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820804 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.821032 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820826 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9psdk\" (UniqueName: \"kubernetes.io/projected/69eab127-a507-40f7-b40a-1ab4967dc8bc-kube-api-access-9psdk\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.821032 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820861 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.821032 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.820919 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.821456 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.821317 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.821456 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.821395 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.822752 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.822604 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.823067 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.823046 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.823153 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.823043 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.823857 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.823814 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.823981 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.823857 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/69eab127-a507-40f7-b40a-1ab4967dc8bc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.824142 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.824106 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.825226 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.825106 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.825321 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.825275 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.825543 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.825523 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-config\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.826452 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.826406 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.826632 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.826614 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.826998 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.826979 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.827046 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.827015 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.827147 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.827124 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-web-config\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.827610 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.827589 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/69eab127-a507-40f7-b40a-1ab4967dc8bc-config-out\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.832060 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.832043 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9psdk\" (UniqueName: \"kubernetes.io/projected/69eab127-a507-40f7-b40a-1ab4967dc8bc-kube-api-access-9psdk\") pod \"prometheus-k8s-0\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:41.888184 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:41.888150 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:42.046273 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:42.046242 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 21:36:42.289343 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:36:42.289305 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69eab127_a507_40f7_b40a_1ab4967dc8bc.slice/crio-d34decc2f77c3a4b588688d9f83532e28d45d0d6da88afdf92cda4df3e388d2c WatchSource:0}: Error finding container d34decc2f77c3a4b588688d9f83532e28d45d0d6da88afdf92cda4df3e388d2c: Status 404 returned error can't find the container with id d34decc2f77c3a4b588688d9f83532e28d45d0d6da88afdf92cda4df3e388d2c Jun 02 21:36:42.622783 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:42.622695 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" event={"ID":"8e77aa53-8d9e-4724-89ac-5e6604cd48a5","Type":"ContainerStarted","Data":"dc674dec8657e32d8eedcd134118873ae16717b08037e989e1261f8e05a2b6ed"} Jun 02 21:36:42.622783 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:42.622733 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" event={"ID":"8e77aa53-8d9e-4724-89ac-5e6604cd48a5","Type":"ContainerStarted","Data":"696300b662cfc7a7c1580319cb96341561f2a77de391e8e134af211c1137d21a"} Jun 02 21:36:42.624047 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:42.624022 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj" event={"ID":"6a49d7a6-3d14-4f12-8d83-8261bce45960","Type":"ContainerStarted","Data":"2332e7d2d995e61f3dc8042697de11fca7acfed3bd337d38a54c7c067ae4e136"} Jun 02 21:36:42.624385 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:42.624354 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj" Jun 02 21:36:42.625185 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:42.625166 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerStarted","Data":"d34decc2f77c3a4b588688d9f83532e28d45d0d6da88afdf92cda4df3e388d2c"} Jun 02 21:36:42.630297 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:42.630238 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj" Jun 02 21:36:42.642545 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:42.642503 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-472dj" podStartSLOduration=0.762731452 podStartE2EDuration="2.642488925s" podCreationTimestamp="2026-06-02 21:36:40 +0000 UTC" firstStartedPulling="2026-06-02 21:36:40.57900274 +0000 UTC m=+177.061587334" lastFinishedPulling="2026-06-02 21:36:42.458760223 +0000 UTC m=+178.941344807" observedRunningTime="2026-06-02 21:36:42.641796556 +0000 UTC m=+179.124381161" watchObservedRunningTime="2026-06-02 21:36:42.642488925 +0000 UTC m=+179.125073528" Jun 02 21:36:43.631029 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:43.630992 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" event={"ID":"8e77aa53-8d9e-4724-89ac-5e6604cd48a5","Type":"ContainerStarted","Data":"3ae6a9e2e5c5f62c41816a29fa76febc622c45c9081e4292f07a47d2792991ac"} Jun 02 21:36:43.631418 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:43.631136 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:43.632557 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:43.632532 2562 generic.go:358] "Generic (PLEG): container finished" podID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerID="c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2" exitCode=0 Jun 02 21:36:43.632678 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:43.632629 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerDied","Data":"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2"} Jun 02 21:36:43.657850 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:43.657808 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" podStartSLOduration=2.068521638 podStartE2EDuration="5.657783111s" podCreationTimestamp="2026-06-02 21:36:38 +0000 UTC" firstStartedPulling="2026-06-02 21:36:38.871054291 +0000 UTC m=+175.353638876" lastFinishedPulling="2026-06-02 21:36:42.460315767 +0000 UTC m=+178.942900349" observedRunningTime="2026-06-02 21:36:43.657336543 +0000 UTC m=+180.139921146" watchObservedRunningTime="2026-06-02 21:36:43.657783111 +0000 UTC m=+180.140367715" Jun 02 21:36:46.618248 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:46.618222 2562 patch_prober.go:28] interesting pod/image-registry-6c6977d9cd-8d7tq container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 02 21:36:46.618519 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:46.618265 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" podUID="2987d45b-7b68-4d07-a82f-01202b7fc38e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 21:36:47.646776 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:47.646736 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerStarted","Data":"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52"} Jun 02 21:36:47.646776 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:47.646780 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerStarted","Data":"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2"} Jun 02 21:36:47.647234 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:47.646794 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerStarted","Data":"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e"} Jun 02 21:36:47.647234 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:47.646807 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerStarted","Data":"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25"} Jun 02 21:36:47.647234 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:47.646819 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerStarted","Data":"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4"} Jun 02 21:36:47.647234 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:47.646830 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerStarted","Data":"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe"} Jun 02 21:36:47.683040 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:47.682993 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.384544055 podStartE2EDuration="6.682980514s" podCreationTimestamp="2026-06-02 21:36:41 +0000 UTC" firstStartedPulling="2026-06-02 21:36:42.291080381 +0000 UTC m=+178.773664962" lastFinishedPulling="2026-06-02 21:36:46.589516836 +0000 UTC m=+183.072101421" observedRunningTime="2026-06-02 21:36:47.680738593 +0000 UTC m=+184.163323196" watchObservedRunningTime="2026-06-02 21:36:47.682980514 +0000 UTC m=+184.165565115" Jun 02 21:36:47.983414 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:47.983392 2562 scope.go:117] "RemoveContainer" containerID="99ce67c461c4e4c52c8c4b0d36c5100822d30a00aa19f86ac34ebecddddd7be0" Jun 02 21:36:48.550667 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:48.550641 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c6977d9cd-8d7tq" Jun 02 21:36:48.650999 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:48.650972 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:36:48.651366 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:48.651090 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" event={"ID":"6c26e207-3abb-40b7-9d56-b3f48d9d3ba4","Type":"ContainerStarted","Data":"2ad2b5334c3ba4279604e3fc04bf56d787439e01a22c26825b21e063c69fa6e9"} Jun 02 21:36:48.680264 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:48.680206 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" podStartSLOduration=50.984571713 podStartE2EDuration="53.680190101s" podCreationTimestamp="2026-06-02 21:35:55 +0000 UTC" firstStartedPulling="2026-06-02 21:35:56.167351125 +0000 UTC m=+132.649935706" lastFinishedPulling="2026-06-02 21:35:58.862969513 +0000 UTC m=+135.345554094" observedRunningTime="2026-06-02 21:36:48.680173089 +0000 UTC m=+185.162757696" watchObservedRunningTime="2026-06-02 21:36:48.680190101 +0000 UTC m=+185.162774707" Jun 02 21:36:49.642787 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:49.642755 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-66bd8fbcc5-rqjtr" Jun 02 21:36:51.888518 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:51.888488 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:36:58.651546 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:58.651513 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:36:58.655069 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:36:58.655046 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-86d7c6cc77-qwjj5" Jun 02 21:37:06.149085 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:06.149050 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-86dd499c55-wbghk_37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce/router/0.log" Jun 02 21:37:06.176888 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:06.176837 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vccz8_68e8e581-ba13-4f0c-8ffe-21bde0160fb7/serve-healthcheck-canary/0.log" Jun 02 21:37:41.888628 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:41.888583 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:37:41.906077 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:41.906051 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:37:42.836739 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:42.836708 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:37:55.787712 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:55.787675 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:37:55.790160 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:55.790140 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4c54dfc-ecd1-466b-90db-eca67b4780c5-metrics-certs\") pod \"network-metrics-daemon-p58r9\" (UID: \"b4c54dfc-ecd1-466b-90db-eca67b4780c5\") " pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:37:55.888609 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:55.888578 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4s2f5\"" Jun 02 21:37:55.895816 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:55.895790 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p58r9" Jun 02 21:37:56.017984 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:56.017958 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p58r9"] Jun 02 21:37:56.021438 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:37:56.021413 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4c54dfc_ecd1_466b_90db_eca67b4780c5.slice/crio-20b950a726cb95952821e51a394e724a704965bec605a83ccdf650378311705c WatchSource:0}: Error finding container 20b950a726cb95952821e51a394e724a704965bec605a83ccdf650378311705c: Status 404 returned error can't find the container with id 20b950a726cb95952821e51a394e724a704965bec605a83ccdf650378311705c Jun 02 21:37:56.870829 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:56.870736 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p58r9" event={"ID":"b4c54dfc-ecd1-466b-90db-eca67b4780c5","Type":"ContainerStarted","Data":"20b950a726cb95952821e51a394e724a704965bec605a83ccdf650378311705c"} Jun 02 21:37:57.875322 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:57.875272 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p58r9" event={"ID":"b4c54dfc-ecd1-466b-90db-eca67b4780c5","Type":"ContainerStarted","Data":"9fb1da6c19d439ad0fa05acfe5bce2618d1a6542e7824f74139a4fb589473699"} Jun 02 21:37:57.875322 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:57.875318 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p58r9" event={"ID":"b4c54dfc-ecd1-466b-90db-eca67b4780c5","Type":"ContainerStarted","Data":"6a2cf897930068aa2017b7e3fbb5b8fa8b3fede37fcbfec1976598d1e2470b4b"} Jun 02 21:37:57.892939 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:57.892893 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p58r9" podStartSLOduration=252.981426618 podStartE2EDuration="4m13.892853497s" podCreationTimestamp="2026-06-02 21:33:44 +0000 UTC" firstStartedPulling="2026-06-02 21:37:56.023466898 +0000 UTC m=+252.506051480" lastFinishedPulling="2026-06-02 21:37:56.934893765 +0000 UTC m=+253.417478359" observedRunningTime="2026-06-02 21:37:57.891652851 +0000 UTC m=+254.374237455" watchObservedRunningTime="2026-06-02 21:37:57.892853497 +0000 UTC m=+254.375438147" Jun 02 21:37:59.913163 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:59.913121 2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 21:37:59.913783 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:59.913745 2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="prometheus" containerID="cri-o://8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe" gracePeriod=600 Jun 02 21:37:59.914131 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:59.913765 2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="kube-rbac-proxy-thanos" containerID="cri-o://0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52" gracePeriod=600 Jun 02 21:37:59.914293 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:59.913775 2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="kube-rbac-proxy-web" containerID="cri-o://048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e" gracePeriod=600 Jun 02 21:37:59.914293 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:59.913797 2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="thanos-sidecar" containerID="cri-o://16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25" gracePeriod=600 Jun 02 21:37:59.914293 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:59.913810 2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="kube-rbac-proxy" containerID="cri-o://17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2" gracePeriod=600 Jun 02 21:37:59.914440 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:37:59.913820 2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="config-reloader" containerID="cri-o://15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4" gracePeriod=600 Jun 02 21:38:00.160002 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.159980 2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:00.223075 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223043 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-serving-certs-ca-bundle\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223221 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223080 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9psdk\" (UniqueName: \"kubernetes.io/projected/69eab127-a507-40f7-b40a-1ab4967dc8bc-kube-api-access-9psdk\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223221 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223108 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223221 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223137 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/69eab127-a507-40f7-b40a-1ab4967dc8bc-config-out\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223221 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223161 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-grpc-tls\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223221 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223199 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223221 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223221 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/69eab127-a507-40f7-b40a-1ab4967dc8bc-tls-assets\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223522 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223244 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-metrics-client-certs\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223522 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223278 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-web-config\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223522 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223307 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-k8s-db\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223522 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223347 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-kube-rbac-proxy\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223522 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223375 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-tls\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223522 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223410 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-k8s-rulefiles-0\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223522 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223433 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:38:00.223522 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223441 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-thanos-prometheus-http-client-file\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223522 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223505 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-metrics-client-ca\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223997 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223564 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-trusted-ca-bundle\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223997 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223595 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-config\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223997 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223731 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-kubelet-serving-ca-bundle\") pod \"69eab127-a507-40f7-b40a-1ab4967dc8bc\" (UID: \"69eab127-a507-40f7-b40a-1ab4967dc8bc\") " Jun 02 21:38:00.223997 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.223992 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.226160 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.224509 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:38:00.226160 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.225342 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:38:00.226160 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.225490 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:38:00.226160 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.225760 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:38:00.226160 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.226111 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:38:00.227110 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.227076 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:38:00.227411 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.227383 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69eab127-a507-40f7-b40a-1ab4967dc8bc-kube-api-access-9psdk" (OuterVolumeSpecName: "kube-api-access-9psdk") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "kube-api-access-9psdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:38:00.227490 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.227462 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69eab127-a507-40f7-b40a-1ab4967dc8bc-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:38:00.227899 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.227820 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69eab127-a507-40f7-b40a-1ab4967dc8bc-config-out" (OuterVolumeSpecName: "config-out") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:38:00.228096 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.228049 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:38:00.228213 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.228148 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:38:00.228213 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.228171 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:38:00.228576 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.228551 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:38:00.228806 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.228774 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:38:00.228949 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.228924 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-config" (OuterVolumeSpecName: "config") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:38:00.229258 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.229236 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:38:00.239518 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.239499 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-web-config" (OuterVolumeSpecName: "web-config") pod "69eab127-a507-40f7-b40a-1ab4967dc8bc" (UID: "69eab127-a507-40f7-b40a-1ab4967dc8bc"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:38:00.324785 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324758 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-config\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.324920 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324786 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.324920 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324803 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9psdk\" (UniqueName: \"kubernetes.io/projected/69eab127-a507-40f7-b40a-1ab4967dc8bc-kube-api-access-9psdk\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.324920 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324819 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.324920 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324836 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/69eab127-a507-40f7-b40a-1ab4967dc8bc-config-out\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.324920 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324850 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-grpc-tls\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.324920 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324865 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.324920 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324905 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/69eab127-a507-40f7-b40a-1ab4967dc8bc-tls-assets\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.324920 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324919 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-metrics-client-certs\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.325217 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324933 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-web-config\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.325217 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324946 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-k8s-db\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.325217 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324960 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-kube-rbac-proxy\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.325217 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324975 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.325217 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.324989 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.325217 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.325004 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/69eab127-a507-40f7-b40a-1ab4967dc8bc-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.325217 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.325019 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-configmap-metrics-client-ca\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.325217 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.325034 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69eab127-a507-40f7-b40a-1ab4967dc8bc-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:38:00.888091 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888052 2562 generic.go:358] "Generic (PLEG): container finished" podID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerID="0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52" exitCode=0 Jun 02 21:38:00.888091 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888087 2562 generic.go:358] "Generic (PLEG): container finished" podID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerID="17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2" exitCode=0 Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888097 2562 generic.go:358] "Generic (PLEG): container finished" podID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerID="048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e" exitCode=0 Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888108 2562 generic.go:358] "Generic (PLEG): container finished" podID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerID="16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25" exitCode=0 Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888117 2562 generic.go:358] "Generic (PLEG): container finished" podID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerID="15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4" exitCode=0 Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888124 2562 generic.go:358] "Generic (PLEG): container finished" podID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerID="8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe" exitCode=0 Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888152 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerDied","Data":"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52"} Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888216 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerDied","Data":"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2"} Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888229 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerDied","Data":"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e"} Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888240 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerDied","Data":"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25"} Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888239 2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888254 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerDied","Data":"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4"} Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888264 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerDied","Data":"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe"} Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888274 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69eab127-a507-40f7-b40a-1ab4967dc8bc","Type":"ContainerDied","Data":"d34decc2f77c3a4b588688d9f83532e28d45d0d6da88afdf92cda4df3e388d2c"} Jun 02 21:38:00.888316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.888291 2562 scope.go:117] "RemoveContainer" containerID="0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52" Jun 02 21:38:00.897593 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.897570 2562 scope.go:117] "RemoveContainer" containerID="17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2" Jun 02 21:38:00.905708 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.905689 2562 scope.go:117] "RemoveContainer" containerID="048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e" Jun 02 21:38:00.913124 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.913062 2562 scope.go:117] "RemoveContainer" containerID="16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25" Jun 02 21:38:00.913993 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.913964 2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 21:38:00.918145 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.918124 2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 21:38:00.924236 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.921612 2562 scope.go:117] "RemoveContainer" containerID="15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4" Jun 02 21:38:00.942227 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942151 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 21:38:00.942293 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942277 2562 scope.go:117] "RemoveContainer" containerID="8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe" Jun 02 21:38:00.942513 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942495 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="kube-rbac-proxy" Jun 02 21:38:00.942578 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942517 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="kube-rbac-proxy" Jun 02 21:38:00.942578 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942529 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="config-reloader" Jun 02 21:38:00.942578 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942535 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="config-reloader" Jun 02 21:38:00.942578 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942543 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="kube-rbac-proxy-web" Jun 02 21:38:00.942578 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942550 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="kube-rbac-proxy-web" Jun 02 21:38:00.942578 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942564 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="prometheus" Jun 02 21:38:00.942578 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942570 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="prometheus" Jun 02 21:38:00.942578 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942576 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="kube-rbac-proxy-thanos" Jun 02 21:38:00.942578 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942581 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="kube-rbac-proxy-thanos" Jun 02 21:38:00.943004 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942597 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="init-config-reloader" Jun 02 21:38:00.943004 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942606 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="init-config-reloader" Jun 02 21:38:00.943004 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942612 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="thanos-sidecar" Jun 02 21:38:00.943004 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942617 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="thanos-sidecar" Jun 02 21:38:00.943004 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942669 2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="config-reloader" Jun 02 21:38:00.943004 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942681 2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="kube-rbac-proxy" Jun 02 21:38:00.943004 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942693 2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="prometheus" Jun 02 21:38:00.943004 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942699 2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="kube-rbac-proxy-web" Jun 02 21:38:00.943004 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942704 2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="kube-rbac-proxy-thanos" Jun 02 21:38:00.943004 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.942710 2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" containerName="thanos-sidecar" Jun 02 21:38:00.947663 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.947645 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:00.950134 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.950115 2562 scope.go:117] "RemoveContainer" containerID="c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2" Jun 02 21:38:00.950211 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.950197 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 02 21:38:00.950273 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.950208 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 02 21:38:00.950273 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.950212 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nqp8j\"" Jun 02 21:38:00.950370 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.950303 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 02 21:38:00.950627 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.950604 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 02 21:38:00.950822 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.950681 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-an31mtppglrsi\"" Jun 02 21:38:00.950822 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.950790 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 02 21:38:00.951026 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.950979 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 02 21:38:00.951124 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.951068 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 02 21:38:00.951188 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.951122 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 02 21:38:00.951292 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.951240 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 02 21:38:00.951345 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.951332 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 02 21:38:00.951649 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.951629 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 02 21:38:00.953809 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.953791 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 02 21:38:00.957684 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.957555 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 02 21:38:00.958110 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.958088 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 21:38:00.959442 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.959426 2562 scope.go:117] "RemoveContainer" containerID="0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52" Jun 02 21:38:00.959751 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:38:00.959725 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": container with ID starting with 0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52 not found: ID does not exist" containerID="0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52" Jun 02 21:38:00.959803 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.959765 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52"} err="failed to get container status \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": rpc error: code = NotFound desc = could not find container \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": container with ID starting with 0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52 not found: ID does not exist" Jun 02 21:38:00.959847 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.959806 2562 scope.go:117] "RemoveContainer" containerID="17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2" Jun 02 21:38:00.960355 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:38:00.960324 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": container with ID starting with 17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2 not found: ID does not exist" containerID="17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2" Jun 02 21:38:00.960431 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.960355 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2"} err="failed to get container status \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": rpc error: code = NotFound desc = could not find container \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": container with ID starting with 17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2 not found: ID does not exist" Jun 02 21:38:00.960431 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.960376 2562 scope.go:117] "RemoveContainer" containerID="048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e" Jun 02 21:38:00.960623 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:38:00.960591 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": container with ID starting with 048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e not found: ID does not exist" containerID="048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e" Jun 02 21:38:00.960679 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.960626 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e"} err="failed to get container status \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": rpc error: code = NotFound desc = could not find container \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": container with ID starting with 048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e not found: ID does not exist" Jun 02 21:38:00.960679 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.960650 2562 scope.go:117] "RemoveContainer" containerID="16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25" Jun 02 21:38:00.960881 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:38:00.960850 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": container with ID starting with 16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25 not found: ID does not exist" containerID="16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25" Jun 02 21:38:00.960946 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.960892 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25"} err="failed to get container status \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": rpc error: code = NotFound desc = could not find container \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": container with ID starting with 16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25 not found: ID does not exist" Jun 02 21:38:00.960946 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.960907 2562 scope.go:117] "RemoveContainer" containerID="15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4" Jun 02 21:38:00.961110 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:38:00.961094 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": container with ID starting with 15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4 not found: ID does not exist" containerID="15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4" Jun 02 21:38:00.961199 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.961111 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4"} err="failed to get container status \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": rpc error: code = NotFound desc = could not find container \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": container with ID starting with 15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4 not found: ID does not exist" Jun 02 21:38:00.961199 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.961122 2562 scope.go:117] "RemoveContainer" containerID="8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe" Jun 02 21:38:00.961308 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:38:00.961287 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": container with ID starting with 8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe not found: ID does not exist" containerID="8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe" Jun 02 21:38:00.961308 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.961300 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe"} err="failed to get container status \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": rpc error: code = NotFound desc = could not find container \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": container with ID starting with 8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe not found: ID does not exist" Jun 02 21:38:00.961382 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.961311 2562 scope.go:117] "RemoveContainer" containerID="c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2" Jun 02 21:38:00.961494 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:38:00.961480 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": container with ID starting with c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2 not found: ID does not exist" containerID="c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2" Jun 02 21:38:00.961539 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.961496 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2"} err="failed to get container status \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": rpc error: code = NotFound desc = could not find container \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": container with ID starting with c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2 not found: ID does not exist" Jun 02 21:38:00.961539 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.961507 2562 scope.go:117] "RemoveContainer" containerID="0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52" Jun 02 21:38:00.961722 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.961707 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52"} err="failed to get container status \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": rpc error: code = NotFound desc = could not find container \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": container with ID starting with 0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52 not found: ID does not exist" Jun 02 21:38:00.961772 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.961722 2562 scope.go:117] "RemoveContainer" containerID="17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2" Jun 02 21:38:00.961936 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.961917 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2"} err="failed to get container status \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": rpc error: code = NotFound desc = could not find container \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": container with ID starting with 17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2 not found: ID does not exist" Jun 02 21:38:00.961974 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.961938 2562 scope.go:117] "RemoveContainer" containerID="048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e" Jun 02 21:38:00.962165 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.962150 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e"} err="failed to get container status \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": rpc error: code = NotFound desc = could not find container \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": container with ID starting with 048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e not found: ID does not exist" Jun 02 21:38:00.962165 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.962165 2562 scope.go:117] "RemoveContainer" containerID="16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25" Jun 02 21:38:00.962366 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.962351 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25"} err="failed to get container status \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": rpc error: code = NotFound desc = could not find container \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": container with ID starting with 16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25 not found: ID does not exist" Jun 02 21:38:00.962366 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.962364 2562 scope.go:117] "RemoveContainer" containerID="15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4" Jun 02 21:38:00.962548 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.962534 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4"} err="failed to get container status \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": rpc error: code = NotFound desc = could not find container \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": container with ID starting with 15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4 not found: ID does not exist" Jun 02 21:38:00.962548 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.962547 2562 scope.go:117] "RemoveContainer" containerID="8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe" Jun 02 21:38:00.962724 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.962710 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe"} err="failed to get container status \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": rpc error: code = NotFound desc = could not find container \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": container with ID starting with 8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe not found: ID does not exist" Jun 02 21:38:00.962770 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.962723 2562 scope.go:117] "RemoveContainer" containerID="c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2" Jun 02 21:38:00.962935 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.962919 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2"} err="failed to get container status \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": rpc error: code = NotFound desc = could not find container \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": container with ID starting with c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2 not found: ID does not exist" Jun 02 21:38:00.962978 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.962936 2562 scope.go:117] "RemoveContainer" containerID="0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52" Jun 02 21:38:00.963155 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.963138 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52"} err="failed to get container status \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": rpc error: code = NotFound desc = could not find container \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": container with ID starting with 0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52 not found: ID does not exist" Jun 02 21:38:00.963155 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.963154 2562 scope.go:117] "RemoveContainer" containerID="17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2" Jun 02 21:38:00.963382 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.963367 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2"} err="failed to get container status \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": rpc error: code = NotFound desc = could not find container \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": container with ID starting with 17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2 not found: ID does not exist" Jun 02 21:38:00.963421 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.963382 2562 scope.go:117] "RemoveContainer" containerID="048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e" Jun 02 21:38:00.963591 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.963575 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e"} err="failed to get container status \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": rpc error: code = NotFound desc = could not find container \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": container with ID starting with 048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e not found: ID does not exist" Jun 02 21:38:00.963631 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.963591 2562 scope.go:117] "RemoveContainer" containerID="16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25" Jun 02 21:38:00.963772 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.963756 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25"} err="failed to get container status \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": rpc error: code = NotFound desc = could not find container \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": container with ID starting with 16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25 not found: ID does not exist" Jun 02 21:38:00.963812 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.963772 2562 scope.go:117] "RemoveContainer" containerID="15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4" Jun 02 21:38:00.963973 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.963958 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4"} err="failed to get container status \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": rpc error: code = NotFound desc = could not find container \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": container with ID starting with 15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4 not found: ID does not exist" Jun 02 21:38:00.963973 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.963973 2562 scope.go:117] "RemoveContainer" containerID="8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe" Jun 02 21:38:00.964194 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.964170 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe"} err="failed to get container status \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": rpc error: code = NotFound desc = could not find container \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": container with ID starting with 8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe not found: ID does not exist" Jun 02 21:38:00.964242 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.964194 2562 scope.go:117] "RemoveContainer" containerID="c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2" Jun 02 21:38:00.964390 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.964375 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2"} err="failed to get container status \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": rpc error: code = NotFound desc = could not find container \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": container with ID starting with c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2 not found: ID does not exist" Jun 02 21:38:00.964433 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.964391 2562 scope.go:117] "RemoveContainer" containerID="0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52" Jun 02 21:38:00.964577 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.964552 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52"} err="failed to get container status \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": rpc error: code = NotFound desc = could not find container \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": container with ID starting with 0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52 not found: ID does not exist" Jun 02 21:38:00.964617 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.964590 2562 scope.go:117] "RemoveContainer" containerID="17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2" Jun 02 21:38:00.964831 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.964816 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2"} err="failed to get container status \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": rpc error: code = NotFound desc = could not find container \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": container with ID starting with 17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2 not found: ID does not exist" Jun 02 21:38:00.964977 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.964832 2562 scope.go:117] "RemoveContainer" containerID="048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e" Jun 02 21:38:00.965088 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.965071 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e"} err="failed to get container status \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": rpc error: code = NotFound desc = could not find container \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": container with ID starting with 048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e not found: ID does not exist" Jun 02 21:38:00.965132 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.965089 2562 scope.go:117] "RemoveContainer" containerID="16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25" Jun 02 21:38:00.965301 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.965285 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25"} err="failed to get container status \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": rpc error: code = NotFound desc = could not find container \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": container with ID starting with 16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25 not found: ID does not exist" Jun 02 21:38:00.965339 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.965303 2562 scope.go:117] "RemoveContainer" containerID="15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4" Jun 02 21:38:00.965525 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.965506 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4"} err="failed to get container status \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": rpc error: code = NotFound desc = could not find container \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": container with ID starting with 15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4 not found: ID does not exist" Jun 02 21:38:00.965562 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.965526 2562 scope.go:117] "RemoveContainer" containerID="8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe" Jun 02 21:38:00.965736 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.965717 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe"} err="failed to get container status \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": rpc error: code = NotFound desc = could not find container \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": container with ID starting with 8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe not found: ID does not exist" Jun 02 21:38:00.965792 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.965738 2562 scope.go:117] "RemoveContainer" containerID="c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2" Jun 02 21:38:00.965966 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.965951 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2"} err="failed to get container status \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": rpc error: code = NotFound desc = could not find container \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": container with ID starting with c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2 not found: ID does not exist" Jun 02 21:38:00.966012 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.965967 2562 scope.go:117] "RemoveContainer" containerID="0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52" Jun 02 21:38:00.966167 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.966151 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52"} err="failed to get container status \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": rpc error: code = NotFound desc = could not find container \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": container with ID starting with 0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52 not found: ID does not exist" Jun 02 21:38:00.966203 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.966168 2562 scope.go:117] "RemoveContainer" containerID="17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2" Jun 02 21:38:00.966366 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.966351 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2"} err="failed to get container status \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": rpc error: code = NotFound desc = could not find container \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": container with ID starting with 17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2 not found: ID does not exist" Jun 02 21:38:00.966405 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.966366 2562 scope.go:117] "RemoveContainer" containerID="048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e" Jun 02 21:38:00.966593 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.966577 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e"} err="failed to get container status \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": rpc error: code = NotFound desc = could not find container \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": container with ID starting with 048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e not found: ID does not exist" Jun 02 21:38:00.966593 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.966593 2562 scope.go:117] "RemoveContainer" containerID="16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25" Jun 02 21:38:00.966813 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.966793 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25"} err="failed to get container status \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": rpc error: code = NotFound desc = could not find container \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": container with ID starting with 16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25 not found: ID does not exist" Jun 02 21:38:00.966813 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.966812 2562 scope.go:117] "RemoveContainer" containerID="15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4" Jun 02 21:38:00.967085 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.967066 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4"} err="failed to get container status \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": rpc error: code = NotFound desc = could not find container \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": container with ID starting with 15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4 not found: ID does not exist" Jun 02 21:38:00.967151 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.967086 2562 scope.go:117] "RemoveContainer" containerID="8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe" Jun 02 21:38:00.967309 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.967287 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe"} err="failed to get container status \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": rpc error: code = NotFound desc = could not find container \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": container with ID starting with 8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe not found: ID does not exist" Jun 02 21:38:00.967369 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.967309 2562 scope.go:117] "RemoveContainer" containerID="c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2" Jun 02 21:38:00.967522 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.967503 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2"} err="failed to get container status \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": rpc error: code = NotFound desc = could not find container \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": container with ID starting with c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2 not found: ID does not exist" Jun 02 21:38:00.967593 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.967521 2562 scope.go:117] "RemoveContainer" containerID="0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52" Jun 02 21:38:00.967752 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.967734 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52"} err="failed to get container status \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": rpc error: code = NotFound desc = could not find container \"0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52\": container with ID starting with 0f85e546c0d344f1f03c6980d5b475a3b99abb3479e2c82bacfcbf51cad32b52 not found: ID does not exist" Jun 02 21:38:00.967820 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.967754 2562 scope.go:117] "RemoveContainer" containerID="17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2" Jun 02 21:38:00.967974 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.967953 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2"} err="failed to get container status \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": rpc error: code = NotFound desc = could not find container \"17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2\": container with ID starting with 17ee639d89be44cdc94170c3b1d10bed624467b1eb54761dcab420d3de8247c2 not found: ID does not exist" Jun 02 21:38:00.968048 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.967974 2562 scope.go:117] "RemoveContainer" containerID="048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e" Jun 02 21:38:00.968200 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.968175 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e"} err="failed to get container status \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": rpc error: code = NotFound desc = could not find container \"048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e\": container with ID starting with 048f745f848546422e781f53d21ddc4eb091d42fa0b4157af6e26351b090fd7e not found: ID does not exist" Jun 02 21:38:00.968276 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.968201 2562 scope.go:117] "RemoveContainer" containerID="16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25" Jun 02 21:38:00.968439 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.968421 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25"} err="failed to get container status \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": rpc error: code = NotFound desc = could not find container \"16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25\": container with ID starting with 16d1af263e38c1caa900c840ba29d2d8431fb95ebbfa413f5e047c4c27e18e25 not found: ID does not exist" Jun 02 21:38:00.968439 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.968439 2562 scope.go:117] "RemoveContainer" containerID="15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4" Jun 02 21:38:00.968650 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.968630 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4"} err="failed to get container status \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": rpc error: code = NotFound desc = could not find container \"15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4\": container with ID starting with 15e0ded4daffb1b10fe9538e97798d3c59f85c41ea4c1c89965e5d37356d4bc4 not found: ID does not exist" Jun 02 21:38:00.968721 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.968650 2562 scope.go:117] "RemoveContainer" containerID="8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe" Jun 02 21:38:00.968890 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.968848 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe"} err="failed to get container status \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": rpc error: code = NotFound desc = could not find container \"8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe\": container with ID starting with 8dc4f376e6028b69409e75dd87199225d79b40e37f470ac9007d4962f837fafe not found: ID does not exist" Jun 02 21:38:00.968966 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.968890 2562 scope.go:117] "RemoveContainer" containerID="c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2" Jun 02 21:38:00.969097 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:00.969081 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2"} err="failed to get container status \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": rpc error: code = NotFound desc = could not find container \"c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2\": container with ID starting with c601a9c57d696ed6477a28f7af650a0a536dcd74dcf5e7ce3f1c4de360802db2 not found: ID does not exist" Jun 02 21:38:01.030302 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030399 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030305 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030399 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030324 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/97348bc0-2989-4116-a11e-483588ec86a6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030399 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030341 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/97348bc0-2989-4116-a11e-483588ec86a6-config-out\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030399 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030360 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030543 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030400 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-config\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030543 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030426 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030543 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030443 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-web-config\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030543 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030465 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030543 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030543 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030511 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030808 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030601 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030808 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030808 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030663 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030808 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030700 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030808 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030729 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/97348bc0-2989-4116-a11e-483588ec86a6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030808 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.030808 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.030805 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2cd6\" (UniqueName: \"kubernetes.io/projected/97348bc0-2989-4116-a11e-483588ec86a6-kube-api-access-z2cd6\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.131984 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.131963 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/97348bc0-2989-4116-a11e-483588ec86a6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132162 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.131990 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132162 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132015 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z2cd6\" (UniqueName: \"kubernetes.io/projected/97348bc0-2989-4116-a11e-483588ec86a6-kube-api-access-z2cd6\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132162 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132031 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132162 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132051 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132162 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132075 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/97348bc0-2989-4116-a11e-483588ec86a6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132162 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132104 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/97348bc0-2989-4116-a11e-483588ec86a6-config-out\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132162 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132127 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132162 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132151 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-config\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132564 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132178 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132564 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132212 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-web-config\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132564 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132253 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132564 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132278 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132564 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132301 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132564 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132338 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132564 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132365 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132564 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132392 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.132564 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132416 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/97348bc0-2989-4116-a11e-483588ec86a6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.133203 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.133014 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.133681 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.133314 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.133681 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.132421 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.133681 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.133417 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.135569 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.135226 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.135569 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.135291 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.135798 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.135777 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/97348bc0-2989-4116-a11e-483588ec86a6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.136533 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.135823 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-web-config\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.136533 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.135826 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/97348bc0-2989-4116-a11e-483588ec86a6-config-out\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.136533 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.135858 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.136796 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.136548 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-config\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.136796 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.136573 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.137221 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.137195 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.137924 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.137898 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.138043 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.138024 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.138349 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.138307 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/97348bc0-2989-4116-a11e-483588ec86a6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.138759 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.138739 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/97348bc0-2989-4116-a11e-483588ec86a6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.140294 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.140273 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2cd6\" (UniqueName: \"kubernetes.io/projected/97348bc0-2989-4116-a11e-483588ec86a6-kube-api-access-z2cd6\") pod \"prometheus-k8s-0\" (UID: \"97348bc0-2989-4116-a11e-483588ec86a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.260621 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.260603 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:01.392930 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.392904 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 21:38:01.394669 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:38:01.394645 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97348bc0_2989_4116_a11e_483588ec86a6.slice/crio-0a99ead16dd204deccdb57936dbf2c236396e051e8c5ddd96dd7a28b5a6e3b23 WatchSource:0}: Error finding container 0a99ead16dd204deccdb57936dbf2c236396e051e8c5ddd96dd7a28b5a6e3b23: Status 404 returned error can't find the container with id 0a99ead16dd204deccdb57936dbf2c236396e051e8c5ddd96dd7a28b5a6e3b23 Jun 02 21:38:01.894255 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.894222 2562 generic.go:358] "Generic (PLEG): container finished" podID="97348bc0-2989-4116-a11e-483588ec86a6" containerID="4f19841b58586a96fb2d1a946555bf668b0f93c65d08e21c921e9891bec81b8f" exitCode=0 Jun 02 21:38:01.894401 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.894308 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"97348bc0-2989-4116-a11e-483588ec86a6","Type":"ContainerDied","Data":"4f19841b58586a96fb2d1a946555bf668b0f93c65d08e21c921e9891bec81b8f"} Jun 02 21:38:01.894401 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.894346 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"97348bc0-2989-4116-a11e-483588ec86a6","Type":"ContainerStarted","Data":"0a99ead16dd204deccdb57936dbf2c236396e051e8c5ddd96dd7a28b5a6e3b23"} Jun 02 21:38:01.992170 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:01.990782 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69eab127-a507-40f7-b40a-1ab4967dc8bc" path="/var/lib/kubelet/pods/69eab127-a507-40f7-b40a-1ab4967dc8bc/volumes" Jun 02 21:38:02.901001 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:02.900963 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"97348bc0-2989-4116-a11e-483588ec86a6","Type":"ContainerStarted","Data":"0f45011ab169c8ca707e937ad49d7e2b1d90560571528fc17a9f0845cfae070f"} Jun 02 21:38:02.901001 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:02.901001 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"97348bc0-2989-4116-a11e-483588ec86a6","Type":"ContainerStarted","Data":"6d53b4d531415d20b89925627dc7377adeed20df42b444e42f1e080fad0940ea"} Jun 02 21:38:02.901218 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:02.901012 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"97348bc0-2989-4116-a11e-483588ec86a6","Type":"ContainerStarted","Data":"6af8706cb5c1d4a1b52479d269b6bfbfda17ac903262af27cc544b49a22cc2c7"} Jun 02 21:38:02.901218 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:02.901022 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"97348bc0-2989-4116-a11e-483588ec86a6","Type":"ContainerStarted","Data":"864f57a1158485ffa76e9eff11d15c1c56a3f414be3a34c8e950288cfcc58069"} Jun 02 21:38:02.901218 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:02.901031 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"97348bc0-2989-4116-a11e-483588ec86a6","Type":"ContainerStarted","Data":"1a96b5a9b5ed57c69889d1556ddb745ceb76e3183b81243503016f83c2f8689d"} Jun 02 21:38:02.901218 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:02.901039 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"97348bc0-2989-4116-a11e-483588ec86a6","Type":"ContainerStarted","Data":"a8cea6cd8578e39ba8563c7a7bceb8de231600269bcd2885f64c5377b2bb78d3"} Jun 02 21:38:02.927798 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:02.927740 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.927720991 podStartE2EDuration="2.927720991s" podCreationTimestamp="2026-06-02 21:38:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:38:02.925224591 +0000 UTC m=+259.407809194" watchObservedRunningTime="2026-06-02 21:38:02.927720991 +0000 UTC m=+259.410305596" Jun 02 21:38:06.261030 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:06.261001 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:38:43.917293 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:43.917268 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:38:43.918178 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:43.918158 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:38:43.926189 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:38:43.926167 2562 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 21:39:01.261178 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:01.261141 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:39:01.276896 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:01.276854 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:39:02.098969 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:02.098945 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 21:39:47.343671 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.343640 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-qgz5w"] Jun 02 21:39:47.347119 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.347103 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgz5w" Jun 02 21:39:47.349351 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.349318 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 21:39:47.352944 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.352919 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qgz5w"] Jun 02 21:39:47.405831 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.405808 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1cd5813c-59f8-4321-8802-6e44db95925d-kubelet-config\") pod \"global-pull-secret-syncer-qgz5w\" (UID: \"1cd5813c-59f8-4321-8802-6e44db95925d\") " pod="kube-system/global-pull-secret-syncer-qgz5w" Jun 02 21:39:47.405959 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.405841 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1cd5813c-59f8-4321-8802-6e44db95925d-dbus\") pod \"global-pull-secret-syncer-qgz5w\" (UID: \"1cd5813c-59f8-4321-8802-6e44db95925d\") " pod="kube-system/global-pull-secret-syncer-qgz5w" Jun 02 21:39:47.405959 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.405885 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd5813c-59f8-4321-8802-6e44db95925d-original-pull-secret\") pod \"global-pull-secret-syncer-qgz5w\" (UID: \"1cd5813c-59f8-4321-8802-6e44db95925d\") " pod="kube-system/global-pull-secret-syncer-qgz5w" Jun 02 21:39:47.506923 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.506888 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1cd5813c-59f8-4321-8802-6e44db95925d-kubelet-config\") pod \"global-pull-secret-syncer-qgz5w\" (UID: \"1cd5813c-59f8-4321-8802-6e44db95925d\") " pod="kube-system/global-pull-secret-syncer-qgz5w" Jun 02 21:39:47.506923 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.506926 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1cd5813c-59f8-4321-8802-6e44db95925d-dbus\") pod \"global-pull-secret-syncer-qgz5w\" (UID: \"1cd5813c-59f8-4321-8802-6e44db95925d\") " pod="kube-system/global-pull-secret-syncer-qgz5w" Jun 02 21:39:47.507054 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.506955 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd5813c-59f8-4321-8802-6e44db95925d-original-pull-secret\") pod \"global-pull-secret-syncer-qgz5w\" (UID: \"1cd5813c-59f8-4321-8802-6e44db95925d\") " pod="kube-system/global-pull-secret-syncer-qgz5w" Jun 02 21:39:47.507054 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.507017 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1cd5813c-59f8-4321-8802-6e44db95925d-kubelet-config\") pod \"global-pull-secret-syncer-qgz5w\" (UID: \"1cd5813c-59f8-4321-8802-6e44db95925d\") " pod="kube-system/global-pull-secret-syncer-qgz5w" Jun 02 21:39:47.507119 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.507102 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1cd5813c-59f8-4321-8802-6e44db95925d-dbus\") pod \"global-pull-secret-syncer-qgz5w\" (UID: \"1cd5813c-59f8-4321-8802-6e44db95925d\") " pod="kube-system/global-pull-secret-syncer-qgz5w" Jun 02 21:39:47.509352 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.509334 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd5813c-59f8-4321-8802-6e44db95925d-original-pull-secret\") pod \"global-pull-secret-syncer-qgz5w\" (UID: \"1cd5813c-59f8-4321-8802-6e44db95925d\") " pod="kube-system/global-pull-secret-syncer-qgz5w" Jun 02 21:39:47.657356 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.657295 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgz5w" Jun 02 21:39:47.785138 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.785108 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qgz5w"] Jun 02 21:39:47.788186 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:39:47.788155 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd5813c_59f8_4321_8802_6e44db95925d.slice/crio-6d8faf9a84df020bd1695823db49de669f610db15c47ef9ad22e67c48083d5ca WatchSource:0}: Error finding container 6d8faf9a84df020bd1695823db49de669f610db15c47ef9ad22e67c48083d5ca: Status 404 returned error can't find the container with id 6d8faf9a84df020bd1695823db49de669f610db15c47ef9ad22e67c48083d5ca Jun 02 21:39:47.789764 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:47.789746 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:39:48.228331 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:48.228296 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qgz5w" event={"ID":"1cd5813c-59f8-4321-8802-6e44db95925d","Type":"ContainerStarted","Data":"6d8faf9a84df020bd1695823db49de669f610db15c47ef9ad22e67c48083d5ca"} Jun 02 21:39:52.246884 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:52.246834 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qgz5w" event={"ID":"1cd5813c-59f8-4321-8802-6e44db95925d","Type":"ContainerStarted","Data":"e999b274ec4e7b8a4286a1fa4df0620d516d31f7c775e899e470b8f63605fe1d"} Jun 02 21:39:52.261682 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:39:52.261638 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qgz5w" podStartSLOduration=1.33082282 podStartE2EDuration="5.261626598s" podCreationTimestamp="2026-06-02 21:39:47 +0000 UTC" firstStartedPulling="2026-06-02 21:39:47.789915967 +0000 UTC m=+364.272500548" lastFinishedPulling="2026-06-02 21:39:51.720719726 +0000 UTC m=+368.203304326" observedRunningTime="2026-06-02 21:39:52.260292014 +0000 UTC m=+368.742876618" watchObservedRunningTime="2026-06-02 21:39:52.261626598 +0000 UTC m=+368.744211227" Jun 02 21:41:24.679922 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.679881 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b"] Jun 02 21:41:24.684701 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.684678 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.687086 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.687061 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 02 21:41:24.687198 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.687102 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 02 21:41:24.687990 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.687976 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 21:41:24.688333 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.688317 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-77t8f\"" Jun 02 21:41:24.688938 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.688923 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 21:41:24.701931 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.701908 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b"] Jun 02 21:41:24.719036 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.719007 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57687145-2417-40e0-b667-8f6b3ac3fb06-webhook-cert\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.719137 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.719056 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/57687145-2417-40e0-b667-8f6b3ac3fb06-manifests\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.719137 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.719090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57687145-2417-40e0-b667-8f6b3ac3fb06-tmp\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.719252 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.719169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/57687145-2417-40e0-b667-8f6b3ac3fb06-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.719252 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.719198 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm54z\" (UniqueName: \"kubernetes.io/projected/57687145-2417-40e0-b667-8f6b3ac3fb06-kube-api-access-qm54z\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.820220 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.820196 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/57687145-2417-40e0-b667-8f6b3ac3fb06-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.820378 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.820225 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qm54z\" (UniqueName: \"kubernetes.io/projected/57687145-2417-40e0-b667-8f6b3ac3fb06-kube-api-access-qm54z\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.820378 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.820248 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57687145-2417-40e0-b667-8f6b3ac3fb06-webhook-cert\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.820378 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.820270 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/57687145-2417-40e0-b667-8f6b3ac3fb06-manifests\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.820560 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.820388 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57687145-2417-40e0-b667-8f6b3ac3fb06-tmp\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.820700 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.820675 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/57687145-2417-40e0-b667-8f6b3ac3fb06-manifests\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.822709 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.822691 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57687145-2417-40e0-b667-8f6b3ac3fb06-tmp\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.822817 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.822783 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57687145-2417-40e0-b667-8f6b3ac3fb06-webhook-cert\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.822883 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.822817 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/57687145-2417-40e0-b667-8f6b3ac3fb06-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.828192 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.828166 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm54z\" (UniqueName: \"kubernetes.io/projected/57687145-2417-40e0-b667-8f6b3ac3fb06-kube-api-access-qm54z\") pod \"opendatahub-operator-controller-manager-8b78d96d9-kb22b\" (UID: \"57687145-2417-40e0-b667-8f6b3ac3fb06\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:24.994385 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:24.994319 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:25.127800 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:25.127564 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b"] Jun 02 21:41:25.518737 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:25.518703 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" event={"ID":"57687145-2417-40e0-b667-8f6b3ac3fb06","Type":"ContainerStarted","Data":"b6817bc3caea47370ff0d8361f6ab7385c775caa7d126761efacde755eccb7b2"} Jun 02 21:41:28.530037 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:28.529965 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" event={"ID":"57687145-2417-40e0-b667-8f6b3ac3fb06","Type":"ContainerStarted","Data":"e4ea5b4a6b55c61bd7685776ded864a0f1a3c898f181ba6fff93e0c081934fb8"} Jun 02 21:41:29.534968 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:29.534889 2562 generic.go:358] "Generic (PLEG): container finished" podID="57687145-2417-40e0-b667-8f6b3ac3fb06" containerID="e4ea5b4a6b55c61bd7685776ded864a0f1a3c898f181ba6fff93e0c081934fb8" exitCode=0 Jun 02 21:41:29.535277 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:29.534972 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" event={"ID":"57687145-2417-40e0-b667-8f6b3ac3fb06","Type":"ContainerDied","Data":"e4ea5b4a6b55c61bd7685776ded864a0f1a3c898f181ba6fff93e0c081934fb8"} Jun 02 21:41:30.540885 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:30.540832 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" event={"ID":"57687145-2417-40e0-b667-8f6b3ac3fb06","Type":"ContainerStarted","Data":"7fb4cd2194b23e744d05180bf9a953ff66a4944de5da54973799d37c368017f2"} Jun 02 21:41:30.541293 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:30.541007 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:30.568468 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:30.568417 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" podStartSLOduration=1.839772312 podStartE2EDuration="6.568401552s" podCreationTimestamp="2026-06-02 21:41:24 +0000 UTC" firstStartedPulling="2026-06-02 21:41:25.13799376 +0000 UTC m=+461.620578341" lastFinishedPulling="2026-06-02 21:41:29.866622986 +0000 UTC m=+466.349207581" observedRunningTime="2026-06-02 21:41:30.565938334 +0000 UTC m=+467.048522937" watchObservedRunningTime="2026-06-02 21:41:30.568401552 +0000 UTC m=+467.050986155" Jun 02 21:41:41.546354 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:41.546321 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-kb22b" Jun 02 21:41:45.208680 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.208643 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj"] Jun 02 21:41:45.217792 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.217771 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.221074 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.221046 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 02 21:41:45.221201 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.221104 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 02 21:41:45.221604 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.221295 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-68gnz\"" Jun 02 21:41:45.221604 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.221401 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 02 21:41:45.221604 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.221463 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 02 21:41:45.221604 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.221599 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:41:45.222224 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.222199 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj"] Jun 02 21:41:45.381515 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.381488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/000e3d93-c163-4468-94a4-3ce3dd2879e2-manager-config\") pod \"lws-controller-manager-78c4cd8bbd-t22jj\" (UID: \"000e3d93-c163-4468-94a4-3ce3dd2879e2\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.381625 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.381527 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/000e3d93-c163-4468-94a4-3ce3dd2879e2-metrics-cert\") pod \"lws-controller-manager-78c4cd8bbd-t22jj\" (UID: \"000e3d93-c163-4468-94a4-3ce3dd2879e2\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.381625 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.381547 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/000e3d93-c163-4468-94a4-3ce3dd2879e2-cert\") pod \"lws-controller-manager-78c4cd8bbd-t22jj\" (UID: \"000e3d93-c163-4468-94a4-3ce3dd2879e2\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.381712 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.381661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4rhh\" (UniqueName: \"kubernetes.io/projected/000e3d93-c163-4468-94a4-3ce3dd2879e2-kube-api-access-z4rhh\") pod \"lws-controller-manager-78c4cd8bbd-t22jj\" (UID: \"000e3d93-c163-4468-94a4-3ce3dd2879e2\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.482863 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.482799 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z4rhh\" (UniqueName: \"kubernetes.io/projected/000e3d93-c163-4468-94a4-3ce3dd2879e2-kube-api-access-z4rhh\") pod \"lws-controller-manager-78c4cd8bbd-t22jj\" (UID: \"000e3d93-c163-4468-94a4-3ce3dd2879e2\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.482863 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.482835 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/000e3d93-c163-4468-94a4-3ce3dd2879e2-manager-config\") pod \"lws-controller-manager-78c4cd8bbd-t22jj\" (UID: \"000e3d93-c163-4468-94a4-3ce3dd2879e2\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.483072 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.482891 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/000e3d93-c163-4468-94a4-3ce3dd2879e2-metrics-cert\") pod \"lws-controller-manager-78c4cd8bbd-t22jj\" (UID: \"000e3d93-c163-4468-94a4-3ce3dd2879e2\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.483072 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.483008 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/000e3d93-c163-4468-94a4-3ce3dd2879e2-cert\") pod \"lws-controller-manager-78c4cd8bbd-t22jj\" (UID: \"000e3d93-c163-4468-94a4-3ce3dd2879e2\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.483538 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.483512 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/000e3d93-c163-4468-94a4-3ce3dd2879e2-manager-config\") pod \"lws-controller-manager-78c4cd8bbd-t22jj\" (UID: \"000e3d93-c163-4468-94a4-3ce3dd2879e2\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.485594 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.485575 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/000e3d93-c163-4468-94a4-3ce3dd2879e2-cert\") pod \"lws-controller-manager-78c4cd8bbd-t22jj\" (UID: \"000e3d93-c163-4468-94a4-3ce3dd2879e2\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.486330 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.486308 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/000e3d93-c163-4468-94a4-3ce3dd2879e2-metrics-cert\") pod \"lws-controller-manager-78c4cd8bbd-t22jj\" (UID: \"000e3d93-c163-4468-94a4-3ce3dd2879e2\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.494349 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.494324 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4rhh\" (UniqueName: \"kubernetes.io/projected/000e3d93-c163-4468-94a4-3ce3dd2879e2-kube-api-access-z4rhh\") pod \"lws-controller-manager-78c4cd8bbd-t22jj\" (UID: \"000e3d93-c163-4468-94a4-3ce3dd2879e2\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.528352 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.528332 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:45.652674 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:45.652642 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj"] Jun 02 21:41:45.656045 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:41:45.656015 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000e3d93_c163_4468_94a4_3ce3dd2879e2.slice/crio-90562e812d9f60be8d68d30c6caeaff340eab75731342762444db2362285a744 WatchSource:0}: Error finding container 90562e812d9f60be8d68d30c6caeaff340eab75731342762444db2362285a744: Status 404 returned error can't find the container with id 90562e812d9f60be8d68d30c6caeaff340eab75731342762444db2362285a744 Jun 02 21:41:46.591163 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:46.591122 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" event={"ID":"000e3d93-c163-4468-94a4-3ce3dd2879e2","Type":"ContainerStarted","Data":"90562e812d9f60be8d68d30c6caeaff340eab75731342762444db2362285a744"} Jun 02 21:41:48.602058 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:48.602024 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" event={"ID":"000e3d93-c163-4468-94a4-3ce3dd2879e2","Type":"ContainerStarted","Data":"d0f22f3aafab8f3d3c21571bcfecbe45699fc4962bb83faf7786e03e0f4215b7"} Jun 02 21:41:48.602456 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:48.602157 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:41:48.624926 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:48.624862 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" podStartSLOduration=1.401128863 podStartE2EDuration="3.624848921s" podCreationTimestamp="2026-06-02 21:41:45 +0000 UTC" firstStartedPulling="2026-06-02 21:41:45.658196232 +0000 UTC m=+482.140780816" lastFinishedPulling="2026-06-02 21:41:47.881916279 +0000 UTC m=+484.364500874" observedRunningTime="2026-06-02 21:41:48.622628149 +0000 UTC m=+485.105212752" watchObservedRunningTime="2026-06-02 21:41:48.624848921 +0000 UTC m=+485.107433524" Jun 02 21:41:59.610287 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:41:59.610252 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-t22jj" Jun 02 21:42:16.448215 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.448181 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz"] Jun 02 21:42:16.451556 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.451538 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.455904 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.455134 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-z8gfv\"" Jun 02 21:42:16.455904 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.455345 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 02 21:42:16.465769 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.465746 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz"] Jun 02 21:42:16.519239 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.519211 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqgm9\" (UniqueName: \"kubernetes.io/projected/90e1e678-129c-40ff-83ad-19c8abcc6101-kube-api-access-sqgm9\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.519337 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.519249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.519337 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.519272 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.519456 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.519357 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.519456 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.519398 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90e1e678-129c-40ff-83ad-19c8abcc6101-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.519562 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.519471 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.519562 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.519512 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.519689 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.519568 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.519689 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.519593 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.620736 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.620711 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sqgm9\" (UniqueName: \"kubernetes.io/projected/90e1e678-129c-40ff-83ad-19c8abcc6101-kube-api-access-sqgm9\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.620838 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.620743 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.620838 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.620763 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.620981 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.620892 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.620981 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.620930 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90e1e678-129c-40ff-83ad-19c8abcc6101-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.620981 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.620959 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.621122 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.620994 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.621122 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.621033 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.621122 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.621058 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.621122 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.621103 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.621305 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.621250 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.621392 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.621345 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.621455 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.621408 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.621747 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.621697 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90e1e678-129c-40ff-83ad-19c8abcc6101-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.623541 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.623515 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.623775 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.623754 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.628549 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.628529 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.629027 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.629004 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqgm9\" (UniqueName: \"kubernetes.io/projected/90e1e678-129c-40ff-83ad-19c8abcc6101-kube-api-access-sqgm9\") pod \"data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.695404 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.695383 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv"] Jun 02 21:42:16.699119 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.699072 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.712087 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.712068 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv"] Jun 02 21:42:16.721825 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.721800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/603e029c-008d-4626-8132-5f65b80272f5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.721923 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.721846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.721923 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.721890 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/603e029c-008d-4626-8132-5f65b80272f5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.722001 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.721955 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.722001 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.721992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/603e029c-008d-4626-8132-5f65b80272f5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.722074 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.722017 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.722074 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.722037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.722074 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.722060 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8s4j\" (UniqueName: \"kubernetes.io/projected/603e029c-008d-4626-8132-5f65b80272f5-kube-api-access-z8s4j\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.722178 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.722086 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.772647 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.772621 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:16.823041 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.823013 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.823172 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.823046 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/603e029c-008d-4626-8132-5f65b80272f5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.823172 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.823094 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.823172 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.823137 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/603e029c-008d-4626-8132-5f65b80272f5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.823339 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.823179 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.823339 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.823203 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.823339 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.823227 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z8s4j\" (UniqueName: \"kubernetes.io/projected/603e029c-008d-4626-8132-5f65b80272f5-kube-api-access-z8s4j\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.823339 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.823259 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.823339 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.823308 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/603e029c-008d-4626-8132-5f65b80272f5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.823592 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.823436 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.823592 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.823573 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.823838 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.823814 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.824047 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.824023 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.824121 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.824050 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/603e029c-008d-4626-8132-5f65b80272f5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.825747 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.825719 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/603e029c-008d-4626-8132-5f65b80272f5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.826225 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.826208 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/603e029c-008d-4626-8132-5f65b80272f5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.832342 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.832143 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8s4j\" (UniqueName: \"kubernetes.io/projected/603e029c-008d-4626-8132-5f65b80272f5-kube-api-access-z8s4j\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.833176 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.833136 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/603e029c-008d-4626-8132-5f65b80272f5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8pdsjv\" (UID: \"603e029c-008d-4626-8132-5f65b80272f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:16.905614 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:16.905592 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz"] Jun 02 21:42:16.907571 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:42:16.907546 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90e1e678_129c_40ff_83ad_19c8abcc6101.slice/crio-69918f29569f36f39c734a37f75b2ece134d135e4ed2d52495d80789e475efff WatchSource:0}: Error finding container 69918f29569f36f39c734a37f75b2ece134d135e4ed2d52495d80789e475efff: Status 404 returned error can't find the container with id 69918f29569f36f39c734a37f75b2ece134d135e4ed2d52495d80789e475efff Jun 02 21:42:17.010126 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:17.010073 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:17.146278 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:17.146256 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv"] Jun 02 21:42:17.147744 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:42:17.147710 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod603e029c_008d_4626_8132_5f65b80272f5.slice/crio-b3277f9deff023e4e541c2afac4071be3af27de0a9f055338275bb6569bf3c30 WatchSource:0}: Error finding container b3277f9deff023e4e541c2afac4071be3af27de0a9f055338275bb6569bf3c30: Status 404 returned error can't find the container with id b3277f9deff023e4e541c2afac4071be3af27de0a9f055338275bb6569bf3c30 Jun 02 21:42:17.706420 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:17.706379 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" event={"ID":"603e029c-008d-4626-8132-5f65b80272f5","Type":"ContainerStarted","Data":"b3277f9deff023e4e541c2afac4071be3af27de0a9f055338275bb6569bf3c30"} Jun 02 21:42:17.707779 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:17.707736 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" event={"ID":"90e1e678-129c-40ff-83ad-19c8abcc6101","Type":"ContainerStarted","Data":"69918f29569f36f39c734a37f75b2ece134d135e4ed2d52495d80789e475efff"} Jun 02 21:42:19.505822 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:19.505775 2562 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 02 21:42:19.506184 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:19.505891 2562 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 02 21:42:19.506184 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:19.505928 2562 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 02 21:42:19.510580 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:19.510553 2562 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 02 21:42:19.510660 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:19.510609 2562 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 02 21:42:19.510660 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:19.510637 2562 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 02 21:42:19.717891 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:19.717815 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" event={"ID":"90e1e678-129c-40ff-83ad-19c8abcc6101","Type":"ContainerStarted","Data":"9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a"} Jun 02 21:42:19.719369 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:19.719339 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" event={"ID":"603e029c-008d-4626-8132-5f65b80272f5","Type":"ContainerStarted","Data":"8f86bbc9755a75f1b73d6bad381625ee9e7106754f978461e0f919d01fe5fbc0"} Jun 02 21:42:19.738055 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:19.738013 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" podStartSLOduration=1.137299308 podStartE2EDuration="3.738000278s" podCreationTimestamp="2026-06-02 21:42:16 +0000 UTC" firstStartedPulling="2026-06-02 21:42:16.909660281 +0000 UTC m=+513.392244862" lastFinishedPulling="2026-06-02 21:42:19.510361234 +0000 UTC m=+515.992945832" observedRunningTime="2026-06-02 21:42:19.735966472 +0000 UTC m=+516.218551075" watchObservedRunningTime="2026-06-02 21:42:19.738000278 +0000 UTC m=+516.220584880" Jun 02 21:42:19.754476 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:19.753975 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" podStartSLOduration=1.3981103369999999 podStartE2EDuration="3.753960744s" podCreationTimestamp="2026-06-02 21:42:16 +0000 UTC" firstStartedPulling="2026-06-02 21:42:17.149634987 +0000 UTC m=+513.632219569" lastFinishedPulling="2026-06-02 21:42:19.505485372 +0000 UTC m=+515.988069976" observedRunningTime="2026-06-02 21:42:19.752456052 +0000 UTC m=+516.235040682" watchObservedRunningTime="2026-06-02 21:42:19.753960744 +0000 UTC m=+516.236545348" Jun 02 21:42:19.773335 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:19.773307 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:20.010462 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.010401 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:20.101664 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.101630 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-5465dccb5c-g98dv"] Jun 02 21:42:20.104993 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.104973 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-5465dccb5c-g98dv" Jun 02 21:42:20.107521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.107496 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rmgsq\"" Jun 02 21:42:20.107636 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.107496 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 02 21:42:20.114195 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.114175 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-5465dccb5c-g98dv"] Jun 02 21:42:20.155831 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.155792 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/7c73f69a-806f-4c44-8fa4-6bcc4639d40a-redirect-config\") pod \"dashboard-redirect-5465dccb5c-g98dv\" (UID: \"7c73f69a-806f-4c44-8fa4-6bcc4639d40a\") " pod="opendatahub/dashboard-redirect-5465dccb5c-g98dv" Jun 02 21:42:20.155983 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.155838 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55pnq\" (UniqueName: \"kubernetes.io/projected/7c73f69a-806f-4c44-8fa4-6bcc4639d40a-kube-api-access-55pnq\") pod \"dashboard-redirect-5465dccb5c-g98dv\" (UID: \"7c73f69a-806f-4c44-8fa4-6bcc4639d40a\") " pod="opendatahub/dashboard-redirect-5465dccb5c-g98dv" Jun 02 21:42:20.174843 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.174816 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:20.256922 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.256888 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/7c73f69a-806f-4c44-8fa4-6bcc4639d40a-redirect-config\") pod \"dashboard-redirect-5465dccb5c-g98dv\" (UID: \"7c73f69a-806f-4c44-8fa4-6bcc4639d40a\") " pod="opendatahub/dashboard-redirect-5465dccb5c-g98dv" Jun 02 21:42:20.257064 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.256958 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-55pnq\" (UniqueName: \"kubernetes.io/projected/7c73f69a-806f-4c44-8fa4-6bcc4639d40a-kube-api-access-55pnq\") pod \"dashboard-redirect-5465dccb5c-g98dv\" (UID: \"7c73f69a-806f-4c44-8fa4-6bcc4639d40a\") " pod="opendatahub/dashboard-redirect-5465dccb5c-g98dv" Jun 02 21:42:20.257534 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.257516 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/7c73f69a-806f-4c44-8fa4-6bcc4639d40a-redirect-config\") pod \"dashboard-redirect-5465dccb5c-g98dv\" (UID: \"7c73f69a-806f-4c44-8fa4-6bcc4639d40a\") " pod="opendatahub/dashboard-redirect-5465dccb5c-g98dv" Jun 02 21:42:20.264493 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.264436 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55pnq\" (UniqueName: \"kubernetes.io/projected/7c73f69a-806f-4c44-8fa4-6bcc4639d40a-kube-api-access-55pnq\") pod \"dashboard-redirect-5465dccb5c-g98dv\" (UID: \"7c73f69a-806f-4c44-8fa4-6bcc4639d40a\") " pod="opendatahub/dashboard-redirect-5465dccb5c-g98dv" Jun 02 21:42:20.418037 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.418012 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-5465dccb5c-g98dv" Jun 02 21:42:20.542483 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.542460 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-5465dccb5c-g98dv"] Jun 02 21:42:20.545032 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:42:20.545004 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c73f69a_806f_4c44_8fa4_6bcc4639d40a.slice/crio-91a3e88ea4e331f6a24391e8e40f07ebffda0beefd9fae75a5d9ac6cf7498042 WatchSource:0}: Error finding container 91a3e88ea4e331f6a24391e8e40f07ebffda0beefd9fae75a5d9ac6cf7498042: Status 404 returned error can't find the container with id 91a3e88ea4e331f6a24391e8e40f07ebffda0beefd9fae75a5d9ac6cf7498042 Jun 02 21:42:20.724236 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.724211 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-5465dccb5c-g98dv" event={"ID":"7c73f69a-806f-4c44-8fa4-6bcc4639d40a","Type":"ContainerStarted","Data":"91a3e88ea4e331f6a24391e8e40f07ebffda0beefd9fae75a5d9ac6cf7498042"} Jun 02 21:42:20.724533 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.724516 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:20.725276 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.725259 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8pdsjv" Jun 02 21:42:20.774160 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.774122 2562 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body= Jun 02 21:42:20.774300 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.774189 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" podUID="90e1e678-129c-40ff-83ad-19c8abcc6101" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" Jun 02 21:42:20.775451 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:20.775427 2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz"] Jun 02 21:42:21.773525 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:21.773465 2562 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body= Jun 02 21:42:21.773525 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:21.773514 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" podUID="90e1e678-129c-40ff-83ad-19c8abcc6101" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" Jun 02 21:42:22.732105 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:22.732049 2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" podUID="90e1e678-129c-40ff-83ad-19c8abcc6101" containerName="istio-proxy" containerID="cri-o://9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a" gracePeriod=30 Jun 02 21:42:23.737022 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:23.736971 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-5465dccb5c-g98dv" event={"ID":"7c73f69a-806f-4c44-8fa4-6bcc4639d40a","Type":"ContainerStarted","Data":"f629cfc12d47d36ee66c2d7b3631f8a6e37f780451ee3a094ee2cedee6bc0fd9"} Jun 02 21:42:23.752986 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:23.752921 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-5465dccb5c-g98dv" podStartSLOduration=1.490437622 podStartE2EDuration="3.752850424s" podCreationTimestamp="2026-06-02 21:42:20 +0000 UTC" firstStartedPulling="2026-06-02 21:42:20.546686435 +0000 UTC m=+517.029271031" lastFinishedPulling="2026-06-02 21:42:22.809099253 +0000 UTC m=+519.291683833" observedRunningTime="2026-06-02 21:42:23.751924572 +0000 UTC m=+520.234509168" watchObservedRunningTime="2026-06-02 21:42:23.752850424 +0000 UTC m=+520.235435028" Jun 02 21:42:27.973380 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:27.973356 2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:28.021208 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021142 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-podinfo\") pod \"90e1e678-129c-40ff-83ad-19c8abcc6101\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " Jun 02 21:42:28.021208 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021191 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-data\") pod \"90e1e678-129c-40ff-83ad-19c8abcc6101\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " Jun 02 21:42:28.021387 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021220 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90e1e678-129c-40ff-83ad-19c8abcc6101-istiod-ca-cert\") pod \"90e1e678-129c-40ff-83ad-19c8abcc6101\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " Jun 02 21:42:28.021387 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021241 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-envoy\") pod \"90e1e678-129c-40ff-83ad-19c8abcc6101\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " Jun 02 21:42:28.021387 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021273 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-workload-certs\") pod \"90e1e678-129c-40ff-83ad-19c8abcc6101\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " Jun 02 21:42:28.021387 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021318 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-token\") pod \"90e1e678-129c-40ff-83ad-19c8abcc6101\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " Jun 02 21:42:28.021387 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021351 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqgm9\" (UniqueName: \"kubernetes.io/projected/90e1e678-129c-40ff-83ad-19c8abcc6101-kube-api-access-sqgm9\") pod \"90e1e678-129c-40ff-83ad-19c8abcc6101\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " Jun 02 21:42:28.021640 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021411 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-credential-socket\") pod \"90e1e678-129c-40ff-83ad-19c8abcc6101\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " Jun 02 21:42:28.021640 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021570 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-data" (OuterVolumeSpecName: "istio-data") pod "90e1e678-129c-40ff-83ad-19c8abcc6101" (UID: "90e1e678-129c-40ff-83ad-19c8abcc6101"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:42:28.021640 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021607 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "90e1e678-129c-40ff-83ad-19c8abcc6101" (UID: "90e1e678-129c-40ff-83ad-19c8abcc6101"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:42:28.021786 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021652 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90e1e678-129c-40ff-83ad-19c8abcc6101-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "90e1e678-129c-40ff-83ad-19c8abcc6101" (UID: "90e1e678-129c-40ff-83ad-19c8abcc6101"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:42:28.021786 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021729 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-workload-socket\") pod \"90e1e678-129c-40ff-83ad-19c8abcc6101\" (UID: \"90e1e678-129c-40ff-83ad-19c8abcc6101\") " Jun 02 21:42:28.022156 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.021963 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "90e1e678-129c-40ff-83ad-19c8abcc6101" (UID: "90e1e678-129c-40ff-83ad-19c8abcc6101"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:42:28.022156 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.022083 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-data\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:42:28.022156 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.022102 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90e1e678-129c-40ff-83ad-19c8abcc6101-istiod-ca-cert\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:42:28.022156 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.022117 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-workload-certs\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:42:28.022156 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.022130 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-credential-socket\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:42:28.022666 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.022272 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "90e1e678-129c-40ff-83ad-19c8abcc6101" (UID: "90e1e678-129c-40ff-83ad-19c8abcc6101"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:42:28.023559 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.023530 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "90e1e678-129c-40ff-83ad-19c8abcc6101" (UID: "90e1e678-129c-40ff-83ad-19c8abcc6101"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 02 21:42:28.023638 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.023619 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "90e1e678-129c-40ff-83ad-19c8abcc6101" (UID: "90e1e678-129c-40ff-83ad-19c8abcc6101"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:42:28.023685 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.023614 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90e1e678-129c-40ff-83ad-19c8abcc6101-kube-api-access-sqgm9" (OuterVolumeSpecName: "kube-api-access-sqgm9") pod "90e1e678-129c-40ff-83ad-19c8abcc6101" (UID: "90e1e678-129c-40ff-83ad-19c8abcc6101"). InnerVolumeSpecName "kube-api-access-sqgm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:42:28.023776 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.023758 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-token" (OuterVolumeSpecName: "istio-token") pod "90e1e678-129c-40ff-83ad-19c8abcc6101" (UID: "90e1e678-129c-40ff-83ad-19c8abcc6101"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:42:28.122989 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.122963 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-token\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:42:28.122989 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.122988 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqgm9\" (UniqueName: \"kubernetes.io/projected/90e1e678-129c-40ff-83ad-19c8abcc6101-kube-api-access-sqgm9\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:42:28.123111 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.122999 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-workload-socket\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:42:28.123111 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.123009 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-podinfo\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:42:28.123111 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.123017 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90e1e678-129c-40ff-83ad-19c8abcc6101-istio-envoy\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:42:28.755826 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.755791 2562 generic.go:358] "Generic (PLEG): container finished" podID="90e1e678-129c-40ff-83ad-19c8abcc6101" containerID="9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a" exitCode=0 Jun 02 21:42:28.756001 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.755894 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" event={"ID":"90e1e678-129c-40ff-83ad-19c8abcc6101","Type":"ContainerDied","Data":"9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a"} Jun 02 21:42:28.756001 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.755915 2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" Jun 02 21:42:28.756001 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.755936 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz" event={"ID":"90e1e678-129c-40ff-83ad-19c8abcc6101","Type":"ContainerDied","Data":"69918f29569f36f39c734a37f75b2ece134d135e4ed2d52495d80789e475efff"} Jun 02 21:42:28.756001 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.755955 2562 scope.go:117] "RemoveContainer" containerID="9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a" Jun 02 21:42:28.764781 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.764765 2562 scope.go:117] "RemoveContainer" containerID="9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a" Jun 02 21:42:28.765081 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:42:28.765061 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a\": container with ID starting with 9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a not found: ID does not exist" containerID="9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a" Jun 02 21:42:28.765142 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.765089 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a"} err="failed to get container status \"9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a\": rpc error: code = NotFound desc = could not find container \"9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a\": container with ID starting with 9154fdece68a24fbf48395f0644377304db2bbaf2ff8261858f14130a70d8d4a not found: ID does not exist" Jun 02 21:42:28.776698 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.776678 2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz"] Jun 02 21:42:28.780550 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:28.780531 2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5dcfdcf8b8qm5kz"] Jun 02 21:42:29.987913 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:29.987851 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90e1e678-129c-40ff-83ad-19c8abcc6101" path="/var/lib/kubelet/pods/90e1e678-129c-40ff-83ad-19c8abcc6101/volumes" Jun 02 21:42:45.130828 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.130799 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sw8mp"] Jun 02 21:42:45.131187 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.131137 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90e1e678-129c-40ff-83ad-19c8abcc6101" containerName="istio-proxy" Jun 02 21:42:45.131187 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.131149 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="90e1e678-129c-40ff-83ad-19c8abcc6101" containerName="istio-proxy" Jun 02 21:42:45.131269 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.131219 2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="90e1e678-129c-40ff-83ad-19c8abcc6101" containerName="istio-proxy" Jun 02 21:42:45.135605 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.135585 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" Jun 02 21:42:45.137851 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.137828 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-xfkvl\"" Jun 02 21:42:45.137967 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.137888 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 21:42:45.138807 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.138791 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 21:42:45.142120 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.142098 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sw8mp"] Jun 02 21:42:45.251491 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.251456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfphn\" (UniqueName: \"kubernetes.io/projected/0fc7063e-0bc3-4d28-b97c-1ffec780e981-kube-api-access-nfphn\") pod \"kuadrant-operator-catalog-sw8mp\" (UID: \"0fc7063e-0bc3-4d28-b97c-1ffec780e981\") " pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" Jun 02 21:42:45.352456 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.352420 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nfphn\" (UniqueName: \"kubernetes.io/projected/0fc7063e-0bc3-4d28-b97c-1ffec780e981-kube-api-access-nfphn\") pod \"kuadrant-operator-catalog-sw8mp\" (UID: \"0fc7063e-0bc3-4d28-b97c-1ffec780e981\") " pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" Jun 02 21:42:45.359996 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.359976 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfphn\" (UniqueName: \"kubernetes.io/projected/0fc7063e-0bc3-4d28-b97c-1ffec780e981-kube-api-access-nfphn\") pod \"kuadrant-operator-catalog-sw8mp\" (UID: \"0fc7063e-0bc3-4d28-b97c-1ffec780e981\") " pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" Jun 02 21:42:45.447479 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.447460 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" Jun 02 21:42:45.504529 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.504498 2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sw8mp"] Jun 02 21:42:45.572393 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.572370 2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sw8mp"] Jun 02 21:42:45.575110 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:42:45.575081 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fc7063e_0bc3_4d28_b97c_1ffec780e981.slice/crio-51cce79b982496ec19e5cda69c30645b8bec26c0a912653e6ef793f97c951b39 WatchSource:0}: Error finding container 51cce79b982496ec19e5cda69c30645b8bec26c0a912653e6ef793f97c951b39: Status 404 returned error can't find the container with id 51cce79b982496ec19e5cda69c30645b8bec26c0a912653e6ef793f97c951b39 Jun 02 21:42:45.711680 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.711621 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4r9m7"] Jun 02 21:42:45.716139 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.716123 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4r9m7" Jun 02 21:42:45.720102 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.720075 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4r9m7"] Jun 02 21:42:45.816316 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.816287 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" event={"ID":"0fc7063e-0bc3-4d28-b97c-1ffec780e981","Type":"ContainerStarted","Data":"51cce79b982496ec19e5cda69c30645b8bec26c0a912653e6ef793f97c951b39"} Jun 02 21:42:45.857811 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.857786 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j56bg\" (UniqueName: \"kubernetes.io/projected/eade3c63-f687-467f-a0f6-aa96a39eac07-kube-api-access-j56bg\") pod \"kuadrant-operator-catalog-4r9m7\" (UID: \"eade3c63-f687-467f-a0f6-aa96a39eac07\") " pod="kuadrant-system/kuadrant-operator-catalog-4r9m7" Jun 02 21:42:45.958619 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.958589 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j56bg\" (UniqueName: \"kubernetes.io/projected/eade3c63-f687-467f-a0f6-aa96a39eac07-kube-api-access-j56bg\") pod \"kuadrant-operator-catalog-4r9m7\" (UID: \"eade3c63-f687-467f-a0f6-aa96a39eac07\") " pod="kuadrant-system/kuadrant-operator-catalog-4r9m7" Jun 02 21:42:45.966236 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:45.966182 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j56bg\" (UniqueName: \"kubernetes.io/projected/eade3c63-f687-467f-a0f6-aa96a39eac07-kube-api-access-j56bg\") pod \"kuadrant-operator-catalog-4r9m7\" (UID: \"eade3c63-f687-467f-a0f6-aa96a39eac07\") " pod="kuadrant-system/kuadrant-operator-catalog-4r9m7" Jun 02 21:42:46.026539 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:46.026513 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4r9m7" Jun 02 21:42:46.164170 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:46.160468 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4r9m7"] Jun 02 21:42:46.196887 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:42:46.196841 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeade3c63_f687_467f_a0f6_aa96a39eac07.slice/crio-302d5c56580d6553401aaaabf95338e3afe77dfbe7e6b7a63ed69936b2d378cf WatchSource:0}: Error finding container 302d5c56580d6553401aaaabf95338e3afe77dfbe7e6b7a63ed69936b2d378cf: Status 404 returned error can't find the container with id 302d5c56580d6553401aaaabf95338e3afe77dfbe7e6b7a63ed69936b2d378cf Jun 02 21:42:46.821324 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:46.821265 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4r9m7" event={"ID":"eade3c63-f687-467f-a0f6-aa96a39eac07","Type":"ContainerStarted","Data":"302d5c56580d6553401aaaabf95338e3afe77dfbe7e6b7a63ed69936b2d378cf"} Jun 02 21:42:47.826010 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:47.825927 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4r9m7" event={"ID":"eade3c63-f687-467f-a0f6-aa96a39eac07","Type":"ContainerStarted","Data":"15b22e273636bee8d6fc7b8d6256cf732b5212a6f52e1ebf735ed20f62d1025e"} Jun 02 21:42:47.827260 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:47.827233 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" event={"ID":"0fc7063e-0bc3-4d28-b97c-1ffec780e981","Type":"ContainerStarted","Data":"ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe"} Jun 02 21:42:47.827376 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:47.827293 2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" podUID="0fc7063e-0bc3-4d28-b97c-1ffec780e981" containerName="registry-server" containerID="cri-o://ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe" gracePeriod=2 Jun 02 21:42:47.843783 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:47.843744 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4r9m7" podStartSLOduration=1.507614868 podStartE2EDuration="2.843732682s" podCreationTimestamp="2026-06-02 21:42:45 +0000 UTC" firstStartedPulling="2026-06-02 21:42:46.198187844 +0000 UTC m=+542.680772425" lastFinishedPulling="2026-06-02 21:42:47.534305655 +0000 UTC m=+544.016890239" observedRunningTime="2026-06-02 21:42:47.842429564 +0000 UTC m=+544.325014169" watchObservedRunningTime="2026-06-02 21:42:47.843732682 +0000 UTC m=+544.326317285" Jun 02 21:42:47.856626 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:47.856582 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" podStartSLOduration=0.937084305 podStartE2EDuration="2.85657081s" podCreationTimestamp="2026-06-02 21:42:45 +0000 UTC" firstStartedPulling="2026-06-02 21:42:45.576511658 +0000 UTC m=+542.059096238" lastFinishedPulling="2026-06-02 21:42:47.495998159 +0000 UTC m=+543.978582743" observedRunningTime="2026-06-02 21:42:47.855992709 +0000 UTC m=+544.338577314" watchObservedRunningTime="2026-06-02 21:42:47.85657081 +0000 UTC m=+544.339155414" Jun 02 21:42:48.069657 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.069633 2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" Jun 02 21:42:48.179684 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.179661 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfphn\" (UniqueName: \"kubernetes.io/projected/0fc7063e-0bc3-4d28-b97c-1ffec780e981-kube-api-access-nfphn\") pod \"0fc7063e-0bc3-4d28-b97c-1ffec780e981\" (UID: \"0fc7063e-0bc3-4d28-b97c-1ffec780e981\") " Jun 02 21:42:48.181887 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.181844 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fc7063e-0bc3-4d28-b97c-1ffec780e981-kube-api-access-nfphn" (OuterVolumeSpecName: "kube-api-access-nfphn") pod "0fc7063e-0bc3-4d28-b97c-1ffec780e981" (UID: "0fc7063e-0bc3-4d28-b97c-1ffec780e981"). InnerVolumeSpecName "kube-api-access-nfphn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:42:48.280417 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.280395 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfphn\" (UniqueName: \"kubernetes.io/projected/0fc7063e-0bc3-4d28-b97c-1ffec780e981-kube-api-access-nfphn\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:42:48.832355 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.832276 2562 generic.go:358] "Generic (PLEG): container finished" podID="0fc7063e-0bc3-4d28-b97c-1ffec780e981" containerID="ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe" exitCode=0 Jun 02 21:42:48.832765 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.832353 2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" Jun 02 21:42:48.832765 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.832368 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" event={"ID":"0fc7063e-0bc3-4d28-b97c-1ffec780e981","Type":"ContainerDied","Data":"ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe"} Jun 02 21:42:48.832765 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.832411 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sw8mp" event={"ID":"0fc7063e-0bc3-4d28-b97c-1ffec780e981","Type":"ContainerDied","Data":"51cce79b982496ec19e5cda69c30645b8bec26c0a912653e6ef793f97c951b39"} Jun 02 21:42:48.832765 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.832427 2562 scope.go:117] "RemoveContainer" containerID="ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe" Jun 02 21:42:48.841981 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.841961 2562 scope.go:117] "RemoveContainer" containerID="ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe" Jun 02 21:42:48.842275 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:42:48.842256 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe\": container with ID starting with ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe not found: ID does not exist" containerID="ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe" Jun 02 21:42:48.842328 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.842283 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe"} err="failed to get container status \"ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe\": rpc error: code = NotFound desc = could not find container \"ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe\": container with ID starting with ee1ebaedbe82f353c936c9218f06baf05eb7c0c899e9a0009113d3c48f0e53fe not found: ID does not exist" Jun 02 21:42:48.853564 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.853537 2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sw8mp"] Jun 02 21:42:48.855541 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:48.855521 2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sw8mp"] Jun 02 21:42:49.987703 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:49.987669 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fc7063e-0bc3-4d28-b97c-1ffec780e981" path="/var/lib/kubelet/pods/0fc7063e-0bc3-4d28-b97c-1ffec780e981/volumes" Jun 02 21:42:56.026814 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:56.026780 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-4r9m7" Jun 02 21:42:56.026814 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:56.026819 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-4r9m7" Jun 02 21:42:56.049969 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:56.049944 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-4r9m7" Jun 02 21:42:56.882220 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:42:56.882194 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-4r9m7" Jun 02 21:43:16.916042 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:16.916006 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx"] Jun 02 21:43:16.916440 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:16.916365 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fc7063e-0bc3-4d28-b97c-1ffec780e981" containerName="registry-server" Jun 02 21:43:16.916440 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:16.916378 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fc7063e-0bc3-4d28-b97c-1ffec780e981" containerName="registry-server" Jun 02 21:43:16.916440 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:16.916437 2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="0fc7063e-0bc3-4d28-b97c-1ffec780e981" containerName="registry-server" Jun 02 21:43:16.919162 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:16.919144 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx" Jun 02 21:43:16.922603 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:16.922579 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jnc5x\"" Jun 02 21:43:16.922732 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:16.922618 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 02 21:43:16.930721 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:16.930700 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx"] Jun 02 21:43:17.003916 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:17.003893 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9ms8\" (UniqueName: \"kubernetes.io/projected/e64d2c72-5346-4272-b56d-4064f80c0598-kube-api-access-m9ms8\") pod \"dns-operator-controller-manager-648d5c98bc-hkhmx\" (UID: \"e64d2c72-5346-4272-b56d-4064f80c0598\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx" Jun 02 21:43:17.104912 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:17.104880 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m9ms8\" (UniqueName: \"kubernetes.io/projected/e64d2c72-5346-4272-b56d-4064f80c0598-kube-api-access-m9ms8\") pod \"dns-operator-controller-manager-648d5c98bc-hkhmx\" (UID: \"e64d2c72-5346-4272-b56d-4064f80c0598\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx" Jun 02 21:43:17.117276 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:17.117255 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9ms8\" (UniqueName: \"kubernetes.io/projected/e64d2c72-5346-4272-b56d-4064f80c0598-kube-api-access-m9ms8\") pod \"dns-operator-controller-manager-648d5c98bc-hkhmx\" (UID: \"e64d2c72-5346-4272-b56d-4064f80c0598\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx" Jun 02 21:43:17.230287 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:17.230262 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx" Jun 02 21:43:17.364039 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:17.364005 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx"] Jun 02 21:43:17.367446 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:43:17.367411 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode64d2c72_5346_4272_b56d_4064f80c0598.slice/crio-5b6508559bc3ffafbdd734efac1ed22813ec6b7e5a29b60d1feb0c09a5c43757 WatchSource:0}: Error finding container 5b6508559bc3ffafbdd734efac1ed22813ec6b7e5a29b60d1feb0c09a5c43757: Status 404 returned error can't find the container with id 5b6508559bc3ffafbdd734efac1ed22813ec6b7e5a29b60d1feb0c09a5c43757 Jun 02 21:43:17.938571 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:17.938533 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx" event={"ID":"e64d2c72-5346-4272-b56d-4064f80c0598","Type":"ContainerStarted","Data":"5b6508559bc3ffafbdd734efac1ed22813ec6b7e5a29b60d1feb0c09a5c43757"} Jun 02 21:43:19.946706 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:19.946675 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx" event={"ID":"e64d2c72-5346-4272-b56d-4064f80c0598","Type":"ContainerStarted","Data":"9daa66a0044f890fb08acd9dea0617d08065f5d83c70ef6d165fe828178c30cb"} Jun 02 21:43:19.947114 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:19.946823 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx" Jun 02 21:43:19.964467 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:19.964419 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx" podStartSLOduration=1.7941869019999999 podStartE2EDuration="3.964408001s" podCreationTimestamp="2026-06-02 21:43:16 +0000 UTC" firstStartedPulling="2026-06-02 21:43:17.369889851 +0000 UTC m=+573.852474440" lastFinishedPulling="2026-06-02 21:43:19.540110958 +0000 UTC m=+576.022695539" observedRunningTime="2026-06-02 21:43:19.961803913 +0000 UTC m=+576.444388515" watchObservedRunningTime="2026-06-02 21:43:19.964408001 +0000 UTC m=+576.446992604" Jun 02 21:43:20.367741 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:20.367670 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m"] Jun 02 21:43:20.375883 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:20.375855 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" Jun 02 21:43:20.378223 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:20.378206 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-m5xcl\"" Jun 02 21:43:20.381569 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:20.381547 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m"] Jun 02 21:43:20.434638 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:20.434614 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k2dd\" (UniqueName: \"kubernetes.io/projected/76a0416f-a315-4f7d-af70-d4887490e559-kube-api-access-6k2dd\") pod \"limitador-operator-controller-manager-85c4996f8c-8lt6m\" (UID: \"76a0416f-a315-4f7d-af70-d4887490e559\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" Jun 02 21:43:20.535091 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:20.535067 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6k2dd\" (UniqueName: \"kubernetes.io/projected/76a0416f-a315-4f7d-af70-d4887490e559-kube-api-access-6k2dd\") pod \"limitador-operator-controller-manager-85c4996f8c-8lt6m\" (UID: \"76a0416f-a315-4f7d-af70-d4887490e559\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" Jun 02 21:43:20.548940 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:20.548914 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k2dd\" (UniqueName: \"kubernetes.io/projected/76a0416f-a315-4f7d-af70-d4887490e559-kube-api-access-6k2dd\") pod \"limitador-operator-controller-manager-85c4996f8c-8lt6m\" (UID: \"76a0416f-a315-4f7d-af70-d4887490e559\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" Jun 02 21:43:20.687362 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:20.687340 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" Jun 02 21:43:20.817231 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:20.817162 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m"] Jun 02 21:43:20.819441 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:43:20.819411 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76a0416f_a315_4f7d_af70_d4887490e559.slice/crio-2367af251b80e801b52685172d068dadf92283a0f03932761703325209b500b4 WatchSource:0}: Error finding container 2367af251b80e801b52685172d068dadf92283a0f03932761703325209b500b4: Status 404 returned error can't find the container with id 2367af251b80e801b52685172d068dadf92283a0f03932761703325209b500b4 Jun 02 21:43:20.951139 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:20.951070 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" event={"ID":"76a0416f-a315-4f7d-af70-d4887490e559","Type":"ContainerStarted","Data":"2367af251b80e801b52685172d068dadf92283a0f03932761703325209b500b4"} Jun 02 21:43:22.960263 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:22.960233 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" event={"ID":"76a0416f-a315-4f7d-af70-d4887490e559","Type":"ContainerStarted","Data":"3c07a46fffcea4036d4615fa9126f3bff4e1383f6b142b64b936609709958e52"} Jun 02 21:43:22.960578 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:22.960335 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" Jun 02 21:43:22.979193 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:22.979148 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" podStartSLOduration=1.00501854 podStartE2EDuration="2.979134563s" podCreationTimestamp="2026-06-02 21:43:20 +0000 UTC" firstStartedPulling="2026-06-02 21:43:20.821425969 +0000 UTC m=+577.304010551" lastFinishedPulling="2026-06-02 21:43:22.795541992 +0000 UTC m=+579.278126574" observedRunningTime="2026-06-02 21:43:22.976545082 +0000 UTC m=+579.459129686" watchObservedRunningTime="2026-06-02 21:43:22.979134563 +0000 UTC m=+579.461719235" Jun 02 21:43:28.463818 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:28.463782 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-vn9bg"] Jun 02 21:43:28.467180 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:28.467163 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-vn9bg" Jun 02 21:43:28.469637 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:28.469615 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-gh6rx\"" Jun 02 21:43:28.477813 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:28.477787 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-vn9bg"] Jun 02 21:43:28.502480 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:28.502455 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g8t2\" (UniqueName: \"kubernetes.io/projected/bc4209c4-437b-48b2-8e52-9952f492f451-kube-api-access-6g8t2\") pod \"authorino-operator-657f44b778-vn9bg\" (UID: \"bc4209c4-437b-48b2-8e52-9952f492f451\") " pod="kuadrant-system/authorino-operator-657f44b778-vn9bg" Jun 02 21:43:28.603837 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:28.603806 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6g8t2\" (UniqueName: \"kubernetes.io/projected/bc4209c4-437b-48b2-8e52-9952f492f451-kube-api-access-6g8t2\") pod \"authorino-operator-657f44b778-vn9bg\" (UID: \"bc4209c4-437b-48b2-8e52-9952f492f451\") " pod="kuadrant-system/authorino-operator-657f44b778-vn9bg" Jun 02 21:43:28.612226 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:28.612194 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g8t2\" (UniqueName: \"kubernetes.io/projected/bc4209c4-437b-48b2-8e52-9952f492f451-kube-api-access-6g8t2\") pod \"authorino-operator-657f44b778-vn9bg\" (UID: \"bc4209c4-437b-48b2-8e52-9952f492f451\") " pod="kuadrant-system/authorino-operator-657f44b778-vn9bg" Jun 02 21:43:28.777787 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:28.777727 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-vn9bg" Jun 02 21:43:28.913413 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:28.913381 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-vn9bg"] Jun 02 21:43:28.916343 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:43:28.916314 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc4209c4_437b_48b2_8e52_9952f492f451.slice/crio-3c4ed8a5fe783f44ba6e2699bc9c911fe2ddbc4a9047e309b759a87af4ab1d73 WatchSource:0}: Error finding container 3c4ed8a5fe783f44ba6e2699bc9c911fe2ddbc4a9047e309b759a87af4ab1d73: Status 404 returned error can't find the container with id 3c4ed8a5fe783f44ba6e2699bc9c911fe2ddbc4a9047e309b759a87af4ab1d73 Jun 02 21:43:28.983672 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:28.983636 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-vn9bg" event={"ID":"bc4209c4-437b-48b2-8e52-9952f492f451","Type":"ContainerStarted","Data":"3c4ed8a5fe783f44ba6e2699bc9c911fe2ddbc4a9047e309b759a87af4ab1d73"} Jun 02 21:43:30.954128 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:30.954096 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hkhmx" Jun 02 21:43:31.994932 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:31.994900 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-vn9bg" event={"ID":"bc4209c4-437b-48b2-8e52-9952f492f451","Type":"ContainerStarted","Data":"d5639af990887c65bd3a0714263782d95c302983f5d9f9372fd49a5154e256cd"} Jun 02 21:43:31.995311 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:31.994985 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-vn9bg" Jun 02 21:43:32.011472 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:32.011361 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-vn9bg" podStartSLOduration=1.821909011 podStartE2EDuration="4.01134745s" podCreationTimestamp="2026-06-02 21:43:28 +0000 UTC" firstStartedPulling="2026-06-02 21:43:28.918479154 +0000 UTC m=+585.401063736" lastFinishedPulling="2026-06-02 21:43:31.107917591 +0000 UTC m=+587.590502175" observedRunningTime="2026-06-02 21:43:32.01118937 +0000 UTC m=+588.493773974" watchObservedRunningTime="2026-06-02 21:43:32.01134745 +0000 UTC m=+588.493932055" Jun 02 21:43:33.967019 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:33.966986 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" Jun 02 21:43:43.001070 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:43.001036 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-vn9bg" Jun 02 21:43:43.949100 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:43.949076 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:43:43.949194 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:43.949077 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:43:44.340668 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.340593 2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m"] Jun 02 21:43:44.341116 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.340825 2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" podUID="76a0416f-a315-4f7d-af70-d4887490e559" containerName="manager" containerID="cri-o://3c07a46fffcea4036d4615fa9126f3bff4e1383f6b142b64b936609709958e52" gracePeriod=2 Jun 02 21:43:44.356773 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.356741 2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m"] Jun 02 21:43:44.365798 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.365773 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss"] Jun 02 21:43:44.366150 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.366136 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76a0416f-a315-4f7d-af70-d4887490e559" containerName="manager" Jun 02 21:43:44.366204 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.366151 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="76a0416f-a315-4f7d-af70-d4887490e559" containerName="manager" Jun 02 21:43:44.366249 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.366211 2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="76a0416f-a315-4f7d-af70-d4887490e559" containerName="manager" Jun 02 21:43:44.371015 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.369744 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss" Jun 02 21:43:44.383469 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.383447 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss"] Jun 02 21:43:44.396510 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.396468 2562 status_manager.go:895] "Failed to get status for pod" podUID="76a0416f-a315-4f7d-af70-d4887490e559" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" err="pods \"limitador-operator-controller-manager-85c4996f8c-8lt6m\" is forbidden: User \"system:node:ip-10-0-128-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-185.ec2.internal' and this object" Jun 02 21:43:44.433546 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.433525 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qghss\" (UniqueName: \"kubernetes.io/projected/590121c9-425d-4ac8-871b-f6e32a48818e-kube-api-access-qghss\") pod \"limitador-operator-controller-manager-85c4996f8c-lt6ss\" (UID: \"590121c9-425d-4ac8-871b-f6e32a48818e\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss" Jun 02 21:43:44.534615 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.534584 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qghss\" (UniqueName: \"kubernetes.io/projected/590121c9-425d-4ac8-871b-f6e32a48818e-kube-api-access-qghss\") pod \"limitador-operator-controller-manager-85c4996f8c-lt6ss\" (UID: \"590121c9-425d-4ac8-871b-f6e32a48818e\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss" Jun 02 21:43:44.544921 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.544898 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qghss\" (UniqueName: \"kubernetes.io/projected/590121c9-425d-4ac8-871b-f6e32a48818e-kube-api-access-qghss\") pod \"limitador-operator-controller-manager-85c4996f8c-lt6ss\" (UID: \"590121c9-425d-4ac8-871b-f6e32a48818e\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss" Jun 02 21:43:44.577249 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.577229 2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" Jun 02 21:43:44.579339 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.579313 2562 status_manager.go:895] "Failed to get status for pod" podUID="76a0416f-a315-4f7d-af70-d4887490e559" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" err="pods \"limitador-operator-controller-manager-85c4996f8c-8lt6m\" is forbidden: User \"system:node:ip-10-0-128-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-185.ec2.internal' and this object" Jun 02 21:43:44.635062 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.635017 2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k2dd\" (UniqueName: \"kubernetes.io/projected/76a0416f-a315-4f7d-af70-d4887490e559-kube-api-access-6k2dd\") pod \"76a0416f-a315-4f7d-af70-d4887490e559\" (UID: \"76a0416f-a315-4f7d-af70-d4887490e559\") " Jun 02 21:43:44.637016 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.636986 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76a0416f-a315-4f7d-af70-d4887490e559-kube-api-access-6k2dd" (OuterVolumeSpecName: "kube-api-access-6k2dd") pod "76a0416f-a315-4f7d-af70-d4887490e559" (UID: "76a0416f-a315-4f7d-af70-d4887490e559"). InnerVolumeSpecName "kube-api-access-6k2dd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:43:44.735804 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.735784 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k2dd\" (UniqueName: \"kubernetes.io/projected/76a0416f-a315-4f7d-af70-d4887490e559-kube-api-access-6k2dd\") on node \"ip-10-0-128-185.ec2.internal\" DevicePath \"\"" Jun 02 21:43:44.742685 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.742663 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss" Jun 02 21:43:44.887121 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:44.886950 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss"] Jun 02 21:43:44.889474 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:43:44.889450 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590121c9_425d_4ac8_871b_f6e32a48818e.slice/crio-9561fd7da44bbabccc618ab577bfdc51002fbc884c4ba6024b17da67e7589bd9 WatchSource:0}: Error finding container 9561fd7da44bbabccc618ab577bfdc51002fbc884c4ba6024b17da67e7589bd9: Status 404 returned error can't find the container with id 9561fd7da44bbabccc618ab577bfdc51002fbc884c4ba6024b17da67e7589bd9 Jun 02 21:43:45.039027 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:45.039000 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss" event={"ID":"590121c9-425d-4ac8-871b-f6e32a48818e","Type":"ContainerStarted","Data":"d39193d18f22e0d57b0257acad5e05b87ca9f5fdeb654388905d36b331550ac8"} Jun 02 21:43:45.039180 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:45.039039 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss" Jun 02 21:43:45.039180 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:45.039056 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss" event={"ID":"590121c9-425d-4ac8-871b-f6e32a48818e","Type":"ContainerStarted","Data":"9561fd7da44bbabccc618ab577bfdc51002fbc884c4ba6024b17da67e7589bd9"} Jun 02 21:43:45.040177 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:45.040151 2562 generic.go:358] "Generic (PLEG): container finished" podID="76a0416f-a315-4f7d-af70-d4887490e559" containerID="3c07a46fffcea4036d4615fa9126f3bff4e1383f6b142b64b936609709958e52" exitCode=0 Jun 02 21:43:45.040263 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:45.040187 2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" Jun 02 21:43:45.040263 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:45.040237 2562 scope.go:117] "RemoveContainer" containerID="3c07a46fffcea4036d4615fa9126f3bff4e1383f6b142b64b936609709958e52" Jun 02 21:43:45.049432 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:45.049414 2562 scope.go:117] "RemoveContainer" containerID="3c07a46fffcea4036d4615fa9126f3bff4e1383f6b142b64b936609709958e52" Jun 02 21:43:45.049714 ip-10-0-128-185 kubenswrapper[2562]: E0602 21:43:45.049687 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c07a46fffcea4036d4615fa9126f3bff4e1383f6b142b64b936609709958e52\": container with ID starting with 3c07a46fffcea4036d4615fa9126f3bff4e1383f6b142b64b936609709958e52 not found: ID does not exist" containerID="3c07a46fffcea4036d4615fa9126f3bff4e1383f6b142b64b936609709958e52" Jun 02 21:43:45.049779 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:45.049727 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c07a46fffcea4036d4615fa9126f3bff4e1383f6b142b64b936609709958e52"} err="failed to get container status \"3c07a46fffcea4036d4615fa9126f3bff4e1383f6b142b64b936609709958e52\": rpc error: code = NotFound desc = could not find container \"3c07a46fffcea4036d4615fa9126f3bff4e1383f6b142b64b936609709958e52\": container with ID starting with 3c07a46fffcea4036d4615fa9126f3bff4e1383f6b142b64b936609709958e52 not found: ID does not exist" Jun 02 21:43:45.057620 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:45.057578 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss" podStartSLOduration=1.057564011 podStartE2EDuration="1.057564011s" podCreationTimestamp="2026-06-02 21:43:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:43:45.056761245 +0000 UTC m=+601.539345848" watchObservedRunningTime="2026-06-02 21:43:45.057564011 +0000 UTC m=+601.540148614" Jun 02 21:43:45.058690 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:45.058667 2562 status_manager.go:895] "Failed to get status for pod" podUID="76a0416f-a315-4f7d-af70-d4887490e559" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" err="pods \"limitador-operator-controller-manager-85c4996f8c-8lt6m\" is forbidden: User \"system:node:ip-10-0-128-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-185.ec2.internal' and this object" Jun 02 21:43:45.060437 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:45.060416 2562 status_manager.go:895] "Failed to get status for pod" podUID="76a0416f-a315-4f7d-af70-d4887490e559" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-8lt6m" err="pods \"limitador-operator-controller-manager-85c4996f8c-8lt6m\" is forbidden: User \"system:node:ip-10-0-128-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-185.ec2.internal' and this object" Jun 02 21:43:45.988072 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:45.988040 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76a0416f-a315-4f7d-af70-d4887490e559" path="/var/lib/kubelet/pods/76a0416f-a315-4f7d-af70-d4887490e559/volumes" Jun 02 21:43:56.048198 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:43:56.048167 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lt6ss" Jun 02 21:44:12.943016 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:12.942971 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf"] Jun 02 21:44:12.946617 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:12.946599 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:12.949130 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:12.949111 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nbl95\"" Jun 02 21:44:12.962754 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:12.962727 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf"] Jun 02 21:44:13.060773 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.060750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.060915 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.060787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-workload-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.060915 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.060806 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b15090-2356-467b-acd5-2310384e88aa-istio-token\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.060915 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.060861 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2h8w\" (UniqueName: \"kubernetes.io/projected/b0b15090-2356-467b-acd5-2310384e88aa-kube-api-access-q2h8w\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.061070 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.060962 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b15090-2356-467b-acd5-2310384e88aa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.061070 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.061014 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-istio-data\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.061131 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.061075 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b15090-2356-467b-acd5-2310384e88aa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.061131 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.061110 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-credential-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.061194 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.061129 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-workload-certs\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.162025 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.161998 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-workload-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.162025 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.162026 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b15090-2356-467b-acd5-2310384e88aa-istio-token\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.162227 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.162048 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q2h8w\" (UniqueName: \"kubernetes.io/projected/b0b15090-2356-467b-acd5-2310384e88aa-kube-api-access-q2h8w\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.162227 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.162079 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b15090-2356-467b-acd5-2310384e88aa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.162227 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.162124 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-istio-data\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.162227 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.162151 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b15090-2356-467b-acd5-2310384e88aa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.162227 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.162185 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-credential-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.162479 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.162228 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-workload-certs\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.162479 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.162374 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.162479 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.162425 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-workload-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.162742 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.162721 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-istio-data\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.163326 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.163084 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-credential-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.163326 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.163263 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-workload-certs\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.164378 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.163327 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b15090-2356-467b-acd5-2310384e88aa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.169012 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.165631 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b15090-2356-467b-acd5-2310384e88aa-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.169012 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.165687 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b15090-2356-467b-acd5-2310384e88aa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.170691 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.170669 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2h8w\" (UniqueName: \"kubernetes.io/projected/b0b15090-2356-467b-acd5-2310384e88aa-kube-api-access-q2h8w\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.172012 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.171995 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b15090-2356-467b-acd5-2310384e88aa-istio-token\") pod \"maas-default-gateway-openshift-default-7df95f575-n7vhf\" (UID: \"b0b15090-2356-467b-acd5-2310384e88aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.258286 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.258231 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:13.388509 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.388473 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf"] Jun 02 21:44:13.391777 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:44:13.391746 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0b15090_2356_467b_acd5_2310384e88aa.slice/crio-b1386a6abda60f307b0600c0c474f73e895399d1ffebcfeac56ebf822187db19 WatchSource:0}: Error finding container b1386a6abda60f307b0600c0c474f73e895399d1ffebcfeac56ebf822187db19: Status 404 returned error can't find the container with id b1386a6abda60f307b0600c0c474f73e895399d1ffebcfeac56ebf822187db19 Jun 02 21:44:13.393969 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.393930 2562 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 02 21:44:13.394054 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.394000 2562 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 02 21:44:13.394054 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:13.394027 2562 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 02 21:44:14.139298 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:14.139261 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" event={"ID":"b0b15090-2356-467b-acd5-2310384e88aa","Type":"ContainerStarted","Data":"b947c7b808192aad073f9470f05d9d12b1c0cac0122031f66ec01aedf2d0952e"} Jun 02 21:44:14.139298 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:14.139294 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" event={"ID":"b0b15090-2356-467b-acd5-2310384e88aa","Type":"ContainerStarted","Data":"b1386a6abda60f307b0600c0c474f73e895399d1ffebcfeac56ebf822187db19"} Jun 02 21:44:14.159575 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:14.159531 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" podStartSLOduration=2.159517622 podStartE2EDuration="2.159517622s" podCreationTimestamp="2026-06-02 21:44:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:44:14.156816308 +0000 UTC m=+630.639400912" watchObservedRunningTime="2026-06-02 21:44:14.159517622 +0000 UTC m=+630.642102225" Jun 02 21:44:14.258713 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:14.258689 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:14.263088 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:14.263070 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:15.142974 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:15.142946 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:15.143929 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:15.143909 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-n7vhf" Jun 02 21:44:29.201286 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.201251 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:44:29.205079 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.205063 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" Jun 02 21:44:29.207321 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.207300 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 02 21:44:29.207411 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.207323 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2ng5x\"" Jun 02 21:44:29.210822 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.210801 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:44:29.236562 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.236539 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:44:29.291361 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.291340 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad6ccb1a-fa11-453d-8697-457dd5c7bb10-config-file\") pod \"limitador-limitador-78c99df468-2rqsd\" (UID: \"ad6ccb1a-fa11-453d-8697-457dd5c7bb10\") " pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" Jun 02 21:44:29.291483 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.291391 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvlwr\" (UniqueName: \"kubernetes.io/projected/ad6ccb1a-fa11-453d-8697-457dd5c7bb10-kube-api-access-gvlwr\") pod \"limitador-limitador-78c99df468-2rqsd\" (UID: \"ad6ccb1a-fa11-453d-8697-457dd5c7bb10\") " pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" Jun 02 21:44:29.391949 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.391908 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad6ccb1a-fa11-453d-8697-457dd5c7bb10-config-file\") pod \"limitador-limitador-78c99df468-2rqsd\" (UID: \"ad6ccb1a-fa11-453d-8697-457dd5c7bb10\") " pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" Jun 02 21:44:29.392057 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.391999 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gvlwr\" (UniqueName: \"kubernetes.io/projected/ad6ccb1a-fa11-453d-8697-457dd5c7bb10-kube-api-access-gvlwr\") pod \"limitador-limitador-78c99df468-2rqsd\" (UID: \"ad6ccb1a-fa11-453d-8697-457dd5c7bb10\") " pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" Jun 02 21:44:29.392460 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.392443 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad6ccb1a-fa11-453d-8697-457dd5c7bb10-config-file\") pod \"limitador-limitador-78c99df468-2rqsd\" (UID: \"ad6ccb1a-fa11-453d-8697-457dd5c7bb10\") " pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" Jun 02 21:44:29.401088 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.401066 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvlwr\" (UniqueName: \"kubernetes.io/projected/ad6ccb1a-fa11-453d-8697-457dd5c7bb10-kube-api-access-gvlwr\") pod \"limitador-limitador-78c99df468-2rqsd\" (UID: \"ad6ccb1a-fa11-453d-8697-457dd5c7bb10\") " pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" Jun 02 21:44:29.517203 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.517149 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" Jun 02 21:44:29.660437 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:29.660414 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:44:29.662919 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:44:29.662889 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad6ccb1a_fa11_453d_8697_457dd5c7bb10.slice/crio-f9754565163a9affce4a0880d475216121704b056581ddec007926ceac432e1b WatchSource:0}: Error finding container f9754565163a9affce4a0880d475216121704b056581ddec007926ceac432e1b: Status 404 returned error can't find the container with id f9754565163a9affce4a0880d475216121704b056581ddec007926ceac432e1b Jun 02 21:44:30.199746 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:30.199703 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" event={"ID":"ad6ccb1a-fa11-453d-8697-457dd5c7bb10","Type":"ContainerStarted","Data":"f9754565163a9affce4a0880d475216121704b056581ddec007926ceac432e1b"} Jun 02 21:44:32.209391 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:32.209361 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" event={"ID":"ad6ccb1a-fa11-453d-8697-457dd5c7bb10","Type":"ContainerStarted","Data":"48a9440de652b84626d80da533e32a65fba924a15d4c1dff7549d4cb8d259261"} Jun 02 21:44:32.209720 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:32.209615 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" Jun 02 21:44:32.228782 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:32.228737 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" podStartSLOduration=0.805640442 podStartE2EDuration="3.228724842s" podCreationTimestamp="2026-06-02 21:44:29 +0000 UTC" firstStartedPulling="2026-06-02 21:44:29.664669253 +0000 UTC m=+646.147253835" lastFinishedPulling="2026-06-02 21:44:32.08775364 +0000 UTC m=+648.570338235" observedRunningTime="2026-06-02 21:44:32.225993778 +0000 UTC m=+648.708578382" watchObservedRunningTime="2026-06-02 21:44:32.228724842 +0000 UTC m=+648.711309502" Jun 02 21:44:43.215314 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:44:43.215282 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-2rqsd" Jun 02 21:46:37.262313 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:46:37.262280 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:47:32.588436 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:47:32.588401 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:47:43.482068 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:47:43.482036 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:48:02.696477 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.696439 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v"] Jun 02 21:48:02.700159 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.700137 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.703818 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.703800 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 02 21:48:02.703818 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.703812 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 02 21:48:02.704020 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.703833 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 02 21:48:02.704020 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.703834 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-h8v7p\"" Jun 02 21:48:02.710334 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.710313 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v"] Jun 02 21:48:02.789804 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.789780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2fd3981-0fc9-4092-8897-c1641979a37e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.789929 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.789816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2fd3981-0fc9-4092-8897-c1641979a37e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.789929 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.789894 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2fd3981-0fc9-4092-8897-c1641979a37e-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.790003 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.789935 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a2fd3981-0fc9-4092-8897-c1641979a37e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.790003 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.789966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2fd3981-0fc9-4092-8897-c1641979a37e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.790083 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.790065 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t87tm\" (UniqueName: \"kubernetes.io/projected/a2fd3981-0fc9-4092-8897-c1641979a37e-kube-api-access-t87tm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.890479 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.890451 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a2fd3981-0fc9-4092-8897-c1641979a37e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.890588 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.890492 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2fd3981-0fc9-4092-8897-c1641979a37e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.890588 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.890540 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t87tm\" (UniqueName: \"kubernetes.io/projected/a2fd3981-0fc9-4092-8897-c1641979a37e-kube-api-access-t87tm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.890588 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.890564 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2fd3981-0fc9-4092-8897-c1641979a37e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.890743 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.890593 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2fd3981-0fc9-4092-8897-c1641979a37e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.890743 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.890633 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2fd3981-0fc9-4092-8897-c1641979a37e-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.890850 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.890829 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a2fd3981-0fc9-4092-8897-c1641979a37e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.890976 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.890958 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2fd3981-0fc9-4092-8897-c1641979a37e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.891033 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.890994 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2fd3981-0fc9-4092-8897-c1641979a37e-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.893110 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.893090 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2fd3981-0fc9-4092-8897-c1641979a37e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.893369 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.893347 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2fd3981-0fc9-4092-8897-c1641979a37e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:02.897944 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:02.897918 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t87tm\" (UniqueName: \"kubernetes.io/projected/a2fd3981-0fc9-4092-8897-c1641979a37e-kube-api-access-t87tm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v\" (UID: \"a2fd3981-0fc9-4092-8897-c1641979a37e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:03.011215 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:03.011163 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:03.135964 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:03.135939 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v"] Jun 02 21:48:03.137849 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:48:03.137822 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2fd3981_0fc9_4092_8897_c1641979a37e.slice/crio-2f4c9e8b22ca9526b9ed5494b3b6394d83b923a1b789107a63576bfa1fee847d WatchSource:0}: Error finding container 2f4c9e8b22ca9526b9ed5494b3b6394d83b923a1b789107a63576bfa1fee847d: Status 404 returned error can't find the container with id 2f4c9e8b22ca9526b9ed5494b3b6394d83b923a1b789107a63576bfa1fee847d Jun 02 21:48:03.139621 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:03.139605 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:48:03.273654 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:03.273599 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:48:03.945123 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:03.945083 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" event={"ID":"a2fd3981-0fc9-4092-8897-c1641979a37e","Type":"ContainerStarted","Data":"2f4c9e8b22ca9526b9ed5494b3b6394d83b923a1b789107a63576bfa1fee847d"} Jun 02 21:48:06.959146 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:06.959112 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" event={"ID":"a2fd3981-0fc9-4092-8897-c1641979a37e","Type":"ContainerStarted","Data":"145cd298f6085b5708ef44afec8b800f9bb68a7e02c2bcb736bb15dd976ecedc"} Jun 02 21:48:08.968579 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:08.968546 2562 generic.go:358] "Generic (PLEG): container finished" podID="a2fd3981-0fc9-4092-8897-c1641979a37e" containerID="145cd298f6085b5708ef44afec8b800f9bb68a7e02c2bcb736bb15dd976ecedc" exitCode=0 Jun 02 21:48:08.969011 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:08.968633 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" event={"ID":"a2fd3981-0fc9-4092-8897-c1641979a37e","Type":"ContainerDied","Data":"145cd298f6085b5708ef44afec8b800f9bb68a7e02c2bcb736bb15dd976ecedc"} Jun 02 21:48:10.978183 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:10.978146 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" event={"ID":"a2fd3981-0fc9-4092-8897-c1641979a37e","Type":"ContainerStarted","Data":"a14fdd86d1793f8400e7e2f994dabff1255ba9d4fcd259302dcff42cbefaaf01"} Jun 02 21:48:10.996571 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:10.996527 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" podStartSLOduration=2.105445569 podStartE2EDuration="8.996515038s" podCreationTimestamp="2026-06-02 21:48:02 +0000 UTC" firstStartedPulling="2026-06-02 21:48:03.139730379 +0000 UTC m=+859.622314960" lastFinishedPulling="2026-06-02 21:48:10.030799844 +0000 UTC m=+866.513384429" observedRunningTime="2026-06-02 21:48:10.994573527 +0000 UTC m=+867.477158131" watchObservedRunningTime="2026-06-02 21:48:10.996515038 +0000 UTC m=+867.479099641" Jun 02 21:48:12.182188 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:12.182148 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:48:13.012292 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:13.012256 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:13.012292 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:13.012296 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:13.024780 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:13.024756 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:14.000573 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:14.000541 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v" Jun 02 21:48:21.483268 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.483231 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk"] Jun 02 21:48:21.499933 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.499901 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk"] Jun 02 21:48:21.500092 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.499969 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.502778 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.502757 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 02 21:48:21.562450 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.562424 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx7rt\" (UniqueName: \"kubernetes.io/projected/72c65d36-10c3-4578-b0cb-cbac94e13809-kube-api-access-cx7rt\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.562561 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.562456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72c65d36-10c3-4578-b0cb-cbac94e13809-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.562561 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.562485 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/72c65d36-10c3-4578-b0cb-cbac94e13809-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.562647 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.562600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72c65d36-10c3-4578-b0cb-cbac94e13809-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.562647 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.562641 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72c65d36-10c3-4578-b0cb-cbac94e13809-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.562721 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.562703 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72c65d36-10c3-4578-b0cb-cbac94e13809-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.663384 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.663347 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/72c65d36-10c3-4578-b0cb-cbac94e13809-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.663521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.663395 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72c65d36-10c3-4578-b0cb-cbac94e13809-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.663521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.663414 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72c65d36-10c3-4578-b0cb-cbac94e13809-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.663521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.663441 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72c65d36-10c3-4578-b0cb-cbac94e13809-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.663521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.663486 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cx7rt\" (UniqueName: \"kubernetes.io/projected/72c65d36-10c3-4578-b0cb-cbac94e13809-kube-api-access-cx7rt\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.663521 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.663517 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72c65d36-10c3-4578-b0cb-cbac94e13809-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.663934 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.663909 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/72c65d36-10c3-4578-b0cb-cbac94e13809-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.664021 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.663937 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72c65d36-10c3-4578-b0cb-cbac94e13809-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.664021 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.663958 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72c65d36-10c3-4578-b0cb-cbac94e13809-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.665803 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.665783 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72c65d36-10c3-4578-b0cb-cbac94e13809-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.665968 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.665952 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72c65d36-10c3-4578-b0cb-cbac94e13809-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.672060 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.672035 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx7rt\" (UniqueName: \"kubernetes.io/projected/72c65d36-10c3-4578-b0cb-cbac94e13809-kube-api-access-cx7rt\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk\" (UID: \"72c65d36-10c3-4578-b0cb-cbac94e13809\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.827169 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.827105 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:21.956140 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:21.956114 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk"] Jun 02 21:48:21.957578 ip-10-0-128-185 kubenswrapper[2562]: W0602 21:48:21.957552 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72c65d36_10c3_4578_b0cb_cbac94e13809.slice/crio-0100abb8991fe411deb109294aa90cdcf88c86614296088c729dd36028987eb9 WatchSource:0}: Error finding container 0100abb8991fe411deb109294aa90cdcf88c86614296088c729dd36028987eb9: Status 404 returned error can't find the container with id 0100abb8991fe411deb109294aa90cdcf88c86614296088c729dd36028987eb9 Jun 02 21:48:22.017928 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:22.017900 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" event={"ID":"72c65d36-10c3-4578-b0cb-cbac94e13809","Type":"ContainerStarted","Data":"54f7d266ca0d3d4b84f14bab0f35e50468f9846eeb2ca4dd6610ab90b4d725ad"} Jun 02 21:48:22.018006 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:22.017938 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" event={"ID":"72c65d36-10c3-4578-b0cb-cbac94e13809","Type":"ContainerStarted","Data":"0100abb8991fe411deb109294aa90cdcf88c86614296088c729dd36028987eb9"} Jun 02 21:48:22.878793 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:22.878758 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:48:24.025448 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:24.025412 2562 generic.go:358] "Generic (PLEG): container finished" podID="72c65d36-10c3-4578-b0cb-cbac94e13809" containerID="54f7d266ca0d3d4b84f14bab0f35e50468f9846eeb2ca4dd6610ab90b4d725ad" exitCode=0 Jun 02 21:48:24.025855 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:24.025484 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" event={"ID":"72c65d36-10c3-4578-b0cb-cbac94e13809","Type":"ContainerDied","Data":"54f7d266ca0d3d4b84f14bab0f35e50468f9846eeb2ca4dd6610ab90b4d725ad"} Jun 02 21:48:25.030942 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:25.030912 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" event={"ID":"72c65d36-10c3-4578-b0cb-cbac94e13809","Type":"ContainerStarted","Data":"41948e8113932664476c570404944ba09eafe99e3eb9a7f988ff83548e049c30"} Jun 02 21:48:25.048486 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:25.048442 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" podStartSLOduration=3.810194456 podStartE2EDuration="4.04842851s" podCreationTimestamp="2026-06-02 21:48:21 +0000 UTC" firstStartedPulling="2026-06-02 21:48:24.02605366 +0000 UTC m=+880.508638241" lastFinishedPulling="2026-06-02 21:48:24.264287712 +0000 UTC m=+880.746872295" observedRunningTime="2026-06-02 21:48:25.047007113 +0000 UTC m=+881.529591719" watchObservedRunningTime="2026-06-02 21:48:25.04842851 +0000 UTC m=+881.531013113" Jun 02 21:48:31.827526 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:31.827446 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:31.827913 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:31.827520 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:31.839785 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:31.839765 2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:32.065085 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:32.065054 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk" Jun 02 21:48:34.385082 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:34.385047 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:48:43.976332 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:43.976304 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:48:43.977794 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:48:43.977773 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:49:55.588943 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:49:55.588903 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:50:01.683448 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:50:01.683370 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:50:07.882226 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:50:07.882187 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:50:19.294958 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:50:19.294927 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:50:28.382557 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:50:28.382519 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:50:37.586899 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:50:37.586853 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:50:46.483747 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:50:46.483711 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:50:57.178159 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:50:57.178123 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:51:28.184603 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:51:28.184521 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:51:34.878801 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:51:34.878760 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:52:12.482904 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:52:12.482853 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:52:27.684687 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:52:27.684649 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:53:05.291755 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:53:05.291674 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:53:22.177105 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:53:22.177065 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:53:37.907853 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:53:37.907819 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:53:44.005201 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:53:44.005168 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:53:44.007854 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:53:44.007830 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:53:53.188599 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:53:53.188561 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:54:51.882070 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:54:51.882036 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:55:00.885132 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:55:00.885090 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:55:17.880860 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:55:17.880821 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:55:26.986186 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:55:26.986140 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:55:44.082066 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:55:44.082036 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:55:52.888029 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:55:52.887994 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:56:23.883182 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:56:23.883106 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:56:32.384126 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:56:32.384083 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:56:42.381202 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:56:42.381168 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:56:50.280379 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:56:50.280341 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:56:58.283499 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:56:58.283467 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:57:16.087227 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:57:16.087193 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:57:28.575900 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:57:28.575807 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:58:15.678571 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:58:15.678536 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:58:22.108305 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:58:22.108276 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:58:28.774502 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:58:28.774460 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:58:37.382368 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:58:37.382331 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:58:44.032106 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:58:44.032076 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:58:44.035104 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:58:44.035083 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 21:58:46.181477 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:58:46.181442 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:58:57.181907 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:58:57.181821 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:59:06.187557 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:59:06.187527 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:59:12.689301 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:59:12.689264 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:59:23.479993 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:59:23.479954 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:59:30.478199 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:59:30.478161 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:59:41.179525 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:59:41.179486 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:59:48.097047 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:59:48.097014 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 21:59:56.782883 ip-10-0-128-185 kubenswrapper[2562]: I0602 21:59:56.782836 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:00:05.517989 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:00:05.517954 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:00:14.291559 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:00:14.291524 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:00:24.787149 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:00:24.787113 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:00:31.483094 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:00:31.483014 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:00:40.277621 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:00:40.277587 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:02:57.185739 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:02:57.185707 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:03:04.081015 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:03:04.080983 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:03:28.379373 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:03:28.379294 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:03:35.477809 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:03:35.477766 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:03:44.058166 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:03:44.058139 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 22:03:44.062543 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:03:44.062518 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 22:03:45.883384 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:03:45.883344 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:03:55.532794 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:03:55.532758 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:04:04.591970 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:04:04.591915 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:04:15.577519 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:04:15.577487 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:04:24.386959 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:04:24.386923 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:04:33.692415 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:04:33.692382 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:04:43.086280 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:04:43.086235 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:04:54.074616 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:04:54.074580 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:05:03.081299 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:05:03.081265 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:05:38.287011 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:05:38.286957 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:06:20.385599 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:06:20.385564 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:06:29.492560 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:06:29.492521 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:06:38.186910 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:06:38.186856 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:06:45.581263 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:06:45.581230 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:06:54.182765 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:06:54.182728 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:07:08.489264 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:07:08.489226 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:07:17.687038 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:07:17.687002 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:07:28.885161 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:07:28.885122 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:07:37.681444 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:07:37.681403 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:07:46.581911 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:07:46.581857 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:07:55.077353 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:07:55.077316 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:08:07.380409 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:08:07.380330 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:08:24.692670 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:08:24.692624 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:08:34.289012 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:08:34.288963 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:08:40.481294 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:08:40.481251 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:08:44.087203 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:08:44.087171 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 22:08:44.091141 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:08:44.091114 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 22:08:48.880987 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:08:48.880959 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:09:08.080664 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:09:08.080624 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:09:16.387022 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:09:16.386985 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:09:25.087690 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:09:25.087652 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:09:31.588502 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:09:31.588466 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:09:42.377615 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:09:42.377582 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:09:49.082210 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:09:49.082169 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:09:59.979954 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:09:59.979914 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:10:12.883461 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:10:12.883429 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:10:21.590155 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:10:21.590122 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:10:34.986533 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:10:34.986495 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:10:44.384518 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:10:44.384487 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:10:52.580862 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:10:52.580830 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:11:03.189099 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:11:03.189016 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:11:11.089736 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:11:11.089699 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:11:27.992711 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:11:27.992681 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:11:37.579704 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:11:37.579662 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:11:46.976173 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:11:46.976139 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:11:57.586712 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:11:57.586675 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:12:20.789625 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:20.789595 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:12:33.776856 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:33.776764 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-2rqsd"] Jun 02 22:12:43.205137 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:43.205082 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-5465dccb5c-g98dv_7c73f69a-806f-4c44-8fa4-6bcc4639d40a/nginx/0.log" Jun 02 22:12:44.411757 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:44.411729 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8b78d96d9-kb22b_57687145-2417-40e0-b667-8f6b3ac3fb06/copy-manifests/0.log" Jun 02 22:12:44.417838 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:44.417817 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8b78d96d9-kb22b_57687145-2417-40e0-b667-8f6b3ac3fb06/manager/0.log" Jun 02 22:12:46.055620 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:46.055588 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-vn9bg_bc4209c4-437b-48b2-8e52-9952f492f451/manager/0.log" Jun 02 22:12:46.165895 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:46.165857 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-hkhmx_e64d2c72-5346-4272-b56d-4064f80c0598/manager/0.log" Jun 02 22:12:46.388524 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:46.388466 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4r9m7_eade3c63-f687-467f-a0f6-aa96a39eac07/registry-server/0.log" Jun 02 22:12:46.629181 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:46.629157 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-2rqsd_ad6ccb1a-fa11-453d-8697-457dd5c7bb10/limitador/0.log" Jun 02 22:12:46.745645 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:46.745617 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-lt6ss_590121c9-425d-4ac8-871b-f6e32a48818e/manager/0.log" Jun 02 22:12:47.076361 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:47.076275 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-595b7776f8pdsjv_603e029c-008d-4626-8132-5f65b80272f5/istio-proxy/0.log" Jun 02 22:12:47.509278 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:47.509247 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7df95f575-n7vhf_b0b15090-2356-467b-acd5-2310384e88aa/istio-proxy/0.log" Jun 02 22:12:47.735907 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:47.735881 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-86dd499c55-wbghk_37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce/router/0.log" Jun 02 22:12:48.293445 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:48.293416 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v_a2fd3981-0fc9-4092-8897-c1641979a37e/storage-initializer/0.log" Jun 02 22:12:48.299627 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:48.299604 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-65h5v_a2fd3981-0fc9-4092-8897-c1641979a37e/main/0.log" Jun 02 22:12:48.523232 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:48.523208 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk_72c65d36-10c3-4578-b0cb-cbac94e13809/storage-initializer/0.log" Jun 02 22:12:48.529440 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:48.529391 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-d2mbk_72c65d36-10c3-4578-b0cb-cbac94e13809/main/0.log" Jun 02 22:12:52.815945 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:52.815913 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6k8gj/must-gather-kcvbn"] Jun 02 22:12:52.819558 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:52.819539 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6k8gj/must-gather-kcvbn" Jun 02 22:12:52.822268 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:52.822248 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6k8gj\"/\"kube-root-ca.crt\"" Jun 02 22:12:52.823779 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:52.823758 2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6k8gj\"/\"openshift-service-ca.crt\"" Jun 02 22:12:52.823896 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:52.823758 2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6k8gj\"/\"default-dockercfg-77j6s\"" Jun 02 22:12:52.838074 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:52.838055 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6k8gj/must-gather-kcvbn"] Jun 02 22:12:52.917614 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:52.917593 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/914b7f50-1fbb-45c2-b0a0-b68afa60fdb6-must-gather-output\") pod \"must-gather-kcvbn\" (UID: \"914b7f50-1fbb-45c2-b0a0-b68afa60fdb6\") " pod="openshift-must-gather-6k8gj/must-gather-kcvbn" Jun 02 22:12:52.917736 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:52.917682 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lth2\" (UniqueName: \"kubernetes.io/projected/914b7f50-1fbb-45c2-b0a0-b68afa60fdb6-kube-api-access-6lth2\") pod \"must-gather-kcvbn\" (UID: \"914b7f50-1fbb-45c2-b0a0-b68afa60fdb6\") " pod="openshift-must-gather-6k8gj/must-gather-kcvbn" Jun 02 22:12:53.018733 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:53.018706 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6lth2\" (UniqueName: \"kubernetes.io/projected/914b7f50-1fbb-45c2-b0a0-b68afa60fdb6-kube-api-access-6lth2\") pod \"must-gather-kcvbn\" (UID: \"914b7f50-1fbb-45c2-b0a0-b68afa60fdb6\") " pod="openshift-must-gather-6k8gj/must-gather-kcvbn" Jun 02 22:12:53.018839 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:53.018767 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/914b7f50-1fbb-45c2-b0a0-b68afa60fdb6-must-gather-output\") pod \"must-gather-kcvbn\" (UID: \"914b7f50-1fbb-45c2-b0a0-b68afa60fdb6\") " pod="openshift-must-gather-6k8gj/must-gather-kcvbn" Jun 02 22:12:53.019069 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:53.019051 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/914b7f50-1fbb-45c2-b0a0-b68afa60fdb6-must-gather-output\") pod \"must-gather-kcvbn\" (UID: \"914b7f50-1fbb-45c2-b0a0-b68afa60fdb6\") " pod="openshift-must-gather-6k8gj/must-gather-kcvbn" Jun 02 22:12:53.026862 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:53.026840 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lth2\" (UniqueName: \"kubernetes.io/projected/914b7f50-1fbb-45c2-b0a0-b68afa60fdb6-kube-api-access-6lth2\") pod \"must-gather-kcvbn\" (UID: \"914b7f50-1fbb-45c2-b0a0-b68afa60fdb6\") " pod="openshift-must-gather-6k8gj/must-gather-kcvbn" Jun 02 22:12:53.140246 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:53.140189 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6k8gj/must-gather-kcvbn" Jun 02 22:12:53.271945 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:53.271905 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6k8gj/must-gather-kcvbn"] Jun 02 22:12:53.274682 ip-10-0-128-185 kubenswrapper[2562]: W0602 22:12:53.274652 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod914b7f50_1fbb_45c2_b0a0_b68afa60fdb6.slice/crio-4d4017ff748d3b9195944646c7fbc01e6cd330d42b14496e2291576da2f448ba WatchSource:0}: Error finding container 4d4017ff748d3b9195944646c7fbc01e6cd330d42b14496e2291576da2f448ba: Status 404 returned error can't find the container with id 4d4017ff748d3b9195944646c7fbc01e6cd330d42b14496e2291576da2f448ba Jun 02 22:12:53.276533 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:53.276516 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 22:12:53.420263 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:53.420182 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6k8gj/must-gather-kcvbn" event={"ID":"914b7f50-1fbb-45c2-b0a0-b68afa60fdb6","Type":"ContainerStarted","Data":"4d4017ff748d3b9195944646c7fbc01e6cd330d42b14496e2291576da2f448ba"} Jun 02 22:12:54.424891 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:54.424825 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6k8gj/must-gather-kcvbn" event={"ID":"914b7f50-1fbb-45c2-b0a0-b68afa60fdb6","Type":"ContainerStarted","Data":"4ddad25c6c5c71fd7455875ab1e83abafaa263432beb42b3f3f08d5be2fc13f4"} Jun 02 22:12:55.432061 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:55.432025 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6k8gj/must-gather-kcvbn" event={"ID":"914b7f50-1fbb-45c2-b0a0-b68afa60fdb6","Type":"ContainerStarted","Data":"6df250a3686327399350d1164ddf16eb137a3170e56385676268f66d36bec0ba"} Jun 02 22:12:55.450485 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:55.450427 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-6k8gj/must-gather-kcvbn" podStartSLOduration=2.484356712 podStartE2EDuration="3.450408006s" podCreationTimestamp="2026-06-02 22:12:52 +0000 UTC" firstStartedPulling="2026-06-02 22:12:53.276645941 +0000 UTC m=+2349.759230523" lastFinishedPulling="2026-06-02 22:12:54.242697233 +0000 UTC m=+2350.725281817" observedRunningTime="2026-06-02 22:12:55.448401295 +0000 UTC m=+2351.930985903" watchObservedRunningTime="2026-06-02 22:12:55.450408006 +0000 UTC m=+2351.932992610" Jun 02 22:12:56.056264 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:56.056234 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qgz5w_1cd5813c-59f8-4321-8802-6e44db95925d/global-pull-secret-syncer/0.log" Jun 02 22:12:56.166368 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:56.166338 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jzc49_cf05a3f2-dd0b-49da-a189-e214dafedee9/konnectivity-agent/0.log" Jun 02 22:12:56.221818 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:12:56.221788 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-185.ec2.internal_aec71be92c8b6b09577cb76075fa990c/haproxy/0.log" Jun 02 22:13:00.468644 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:00.468608 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-vn9bg_bc4209c4-437b-48b2-8e52-9952f492f451/manager/0.log" Jun 02 22:13:00.495063 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:00.495001 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-hkhmx_e64d2c72-5346-4272-b56d-4064f80c0598/manager/0.log" Jun 02 22:13:00.552131 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:00.552093 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4r9m7_eade3c63-f687-467f-a0f6-aa96a39eac07/registry-server/0.log" Jun 02 22:13:00.653836 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:00.653804 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-2rqsd_ad6ccb1a-fa11-453d-8697-457dd5c7bb10/limitador/0.log" Jun 02 22:13:00.725237 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:00.725126 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-lt6ss_590121c9-425d-4ac8-871b-f6e32a48818e/manager/0.log" Jun 02 22:13:02.246317 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:02.246235 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-69b6cbdb88-wdn2r_e6f2d34a-e816-4c95-9068-359e6d8f43ff/cluster-monitoring-operator/0.log" Jun 02 22:13:02.285921 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:02.285885 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-wkw9b_b79a288e-785b-4ac6-b08d-fdc03727a420/kube-state-metrics/0.log" Jun 02 22:13:02.319828 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:02.319797 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-wkw9b_b79a288e-785b-4ac6-b08d-fdc03727a420/kube-rbac-proxy-main/0.log" Jun 02 22:13:02.356500 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:02.356449 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-wkw9b_b79a288e-785b-4ac6-b08d-fdc03727a420/kube-rbac-proxy-self/0.log" Jun 02 22:13:02.435786 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:02.435756 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-472dj_6a49d7a6-3d14-4f12-8d83-8261bce45960/monitoring-plugin/0.log" Jun 02 22:13:02.598392 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:02.598361 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9pt47_e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec/node-exporter/0.log" Jun 02 22:13:02.638141 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:02.638112 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9pt47_e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec/kube-rbac-proxy/0.log" Jun 02 22:13:02.691956 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:02.691839 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9pt47_e4742e5f-abfc-4e35-adf6-a8a6aa42c7ec/init-textfile/0.log" Jun 02 22:13:02.907890 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:02.907380 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-xh8k8_290ed1fe-5e18-451c-9825-f43cf21909e4/kube-rbac-proxy-main/0.log" Jun 02 22:13:02.958529 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:02.958454 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-xh8k8_290ed1fe-5e18-451c-9825-f43cf21909e4/kube-rbac-proxy-self/0.log" Jun 02 22:13:02.997912 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:02.997880 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-xh8k8_290ed1fe-5e18-451c-9825-f43cf21909e4/openshift-state-metrics/0.log" Jun 02 22:13:03.050126 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.050095 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_97348bc0-2989-4116-a11e-483588ec86a6/prometheus/0.log" Jun 02 22:13:03.111353 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.111278 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_97348bc0-2989-4116-a11e-483588ec86a6/config-reloader/0.log" Jun 02 22:13:03.138592 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.138565 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_97348bc0-2989-4116-a11e-483588ec86a6/thanos-sidecar/0.log" Jun 02 22:13:03.164168 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.164137 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_97348bc0-2989-4116-a11e-483588ec86a6/kube-rbac-proxy-web/0.log" Jun 02 22:13:03.191453 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.191428 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_97348bc0-2989-4116-a11e-483588ec86a6/kube-rbac-proxy/0.log" Jun 02 22:13:03.213740 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.213708 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_97348bc0-2989-4116-a11e-483588ec86a6/kube-rbac-proxy-thanos/0.log" Jun 02 22:13:03.237272 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.237242 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_97348bc0-2989-4116-a11e-483588ec86a6/init-config-reloader/0.log" Jun 02 22:13:03.402609 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.402531 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bd8fbcc5-rqjtr_8e77aa53-8d9e-4724-89ac-5e6604cd48a5/thanos-query/0.log" Jun 02 22:13:03.425988 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.425961 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bd8fbcc5-rqjtr_8e77aa53-8d9e-4724-89ac-5e6604cd48a5/kube-rbac-proxy-web/0.log" Jun 02 22:13:03.444786 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.444753 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bd8fbcc5-rqjtr_8e77aa53-8d9e-4724-89ac-5e6604cd48a5/kube-rbac-proxy/0.log" Jun 02 22:13:03.463351 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.463321 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bd8fbcc5-rqjtr_8e77aa53-8d9e-4724-89ac-5e6604cd48a5/prom-label-proxy/0.log" Jun 02 22:13:03.484718 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.484662 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bd8fbcc5-rqjtr_8e77aa53-8d9e-4724-89ac-5e6604cd48a5/kube-rbac-proxy-rules/0.log" Jun 02 22:13:03.502450 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:03.502421 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bd8fbcc5-rqjtr_8e77aa53-8d9e-4724-89ac-5e6604cd48a5/kube-rbac-proxy-metrics/0.log" Jun 02 22:13:05.209105 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:05.209070 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/2.log" Jun 02 22:13:05.216022 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:05.215998 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-qwjj5_6c26e207-3abb-40b7-9d56-b3f48d9d3ba4/console-operator/3.log" Jun 02 22:13:07.052604 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.052571 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bs2gf_d51d5fac-3da4-4fb8-a84d-6c9e748a91af/dns/0.log" Jun 02 22:13:07.072899 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.072845 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bs2gf_d51d5fac-3da4-4fb8-a84d-6c9e748a91af/kube-rbac-proxy/0.log" Jun 02 22:13:07.214928 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.214904 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zvfhn_b44e90a0-109c-481c-bb97-0e077753e751/dns-node-resolver/0.log" Jun 02 22:13:07.294231 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.294190 2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm"] Jun 02 22:13:07.305297 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.305215 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm"] Jun 02 22:13:07.305440 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.305344 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.379104 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.379068 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91666e09-1ee1-48c4-ae5a-e970f214266c-lib-modules\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.379445 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.379417 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svt6f\" (UniqueName: \"kubernetes.io/projected/91666e09-1ee1-48c4-ae5a-e970f214266c-kube-api-access-svt6f\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.379689 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.379672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/91666e09-1ee1-48c4-ae5a-e970f214266c-proc\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.379856 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.379840 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91666e09-1ee1-48c4-ae5a-e970f214266c-sys\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.380046 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.380028 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/91666e09-1ee1-48c4-ae5a-e970f214266c-podres\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.481483 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.481451 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/91666e09-1ee1-48c4-ae5a-e970f214266c-proc\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.481642 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.481489 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91666e09-1ee1-48c4-ae5a-e970f214266c-sys\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.481642 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.481517 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/91666e09-1ee1-48c4-ae5a-e970f214266c-podres\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.481642 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.481563 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91666e09-1ee1-48c4-ae5a-e970f214266c-lib-modules\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.481642 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.481569 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/91666e09-1ee1-48c4-ae5a-e970f214266c-proc\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.481642 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.481598 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91666e09-1ee1-48c4-ae5a-e970f214266c-sys\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.481642 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.481637 2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-svt6f\" (UniqueName: \"kubernetes.io/projected/91666e09-1ee1-48c4-ae5a-e970f214266c-kube-api-access-svt6f\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.481841 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.481688 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/91666e09-1ee1-48c4-ae5a-e970f214266c-podres\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.481841 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.481718 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91666e09-1ee1-48c4-ae5a-e970f214266c-lib-modules\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.497371 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.497344 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svt6f\" (UniqueName: \"kubernetes.io/projected/91666e09-1ee1-48c4-ae5a-e970f214266c-kube-api-access-svt6f\") pod \"perf-node-gather-daemonset-hxrsm\" (UID: \"91666e09-1ee1-48c4-ae5a-e970f214266c\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.619335 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.619263 2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:07.775082 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.775053 2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm"] Jun 02 22:13:07.775854 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.775835 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c6977d9cd-8d7tq_2987d45b-7b68-4d07-a82f-01202b7fc38e/registry/0.log" Jun 02 22:13:07.778212 ip-10-0-128-185 kubenswrapper[2562]: W0602 22:13:07.778184 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod91666e09_1ee1_48c4_ae5a_e970f214266c.slice/crio-c5d8658a46bdb8bc2d61e1705914a0c7021e6984459c0106178aa5ecc9d2b13c WatchSource:0}: Error finding container c5d8658a46bdb8bc2d61e1705914a0c7021e6984459c0106178aa5ecc9d2b13c: Status 404 returned error can't find the container with id c5d8658a46bdb8bc2d61e1705914a0c7021e6984459c0106178aa5ecc9d2b13c Jun 02 22:13:07.815903 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:07.815882 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8v5xt_c1e9b389-999a-4410-8048-ff2e03df62dc/node-ca/0.log" Jun 02 22:13:08.492556 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:08.492514 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" event={"ID":"91666e09-1ee1-48c4-ae5a-e970f214266c","Type":"ContainerStarted","Data":"8f21fc99f65c0cb6dd5670d2b6acb44562d6f3a1bbf7774d54ab9e6dda32a7c8"} Jun 02 22:13:08.492975 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:08.492560 2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" event={"ID":"91666e09-1ee1-48c4-ae5a-e970f214266c","Type":"ContainerStarted","Data":"c5d8658a46bdb8bc2d61e1705914a0c7021e6984459c0106178aa5ecc9d2b13c"} Jun 02 22:13:08.492975 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:08.492650 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:08.509337 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:08.509286 2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" podStartSLOduration=1.509266153 podStartE2EDuration="1.509266153s" podCreationTimestamp="2026-06-02 22:13:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 22:13:08.506742188 +0000 UTC m=+2364.989326792" watchObservedRunningTime="2026-06-02 22:13:08.509266153 +0000 UTC m=+2364.991850757" Jun 02 22:13:08.621153 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:08.621125 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-595b7776f8pdsjv_603e029c-008d-4626-8132-5f65b80272f5/istio-proxy/0.log" Jun 02 22:13:08.803693 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:08.803620 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7df95f575-n7vhf_b0b15090-2356-467b-acd5-2310384e88aa/istio-proxy/0.log" Jun 02 22:13:08.881048 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:08.881022 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-86dd499c55-wbghk_37472cf6-e7ec-41ff-9ae7-3f54e7cfe4ce/router/0.log" Jun 02 22:13:09.412373 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:09.412347 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vccz8_68e8e581-ba13-4f0c-8ffe-21bde0160fb7/serve-healthcheck-canary/0.log" Jun 02 22:13:09.919765 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:09.919736 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lcgft_1f20a8a4-7f67-478e-a561-ae04f286726a/kube-rbac-proxy/0.log" Jun 02 22:13:09.939096 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:09.939073 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lcgft_1f20a8a4-7f67-478e-a561-ae04f286726a/exporter/0.log" Jun 02 22:13:09.964917 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:09.964893 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lcgft_1f20a8a4-7f67-478e-a561-ae04f286726a/extractor/0.log" Jun 02 22:13:12.034797 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:12.034772 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-5465dccb5c-g98dv_7c73f69a-806f-4c44-8fa4-6bcc4639d40a/nginx/0.log" Jun 02 22:13:12.392394 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:12.392321 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8b78d96d9-kb22b_57687145-2417-40e0-b667-8f6b3ac3fb06/manager/0.log" Jun 02 22:13:12.408482 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:12.408459 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8b78d96d9-kb22b_57687145-2417-40e0-b667-8f6b3ac3fb06/copy-manifests/0.log" Jun 02 22:13:13.798728 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:13.798697 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78c4cd8bbd-t22jj_000e3d93-c163-4468-94a4-3ce3dd2879e2/manager/0.log" Jun 02 22:13:14.508712 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:14.508685 2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-hxrsm" Jun 02 22:13:18.473395 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:18.473314 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-w72h2_40456ffe-4d23-4e29-8a3d-845996885da8/migrator/0.log" Jun 02 22:13:18.493167 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:18.493142 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-w72h2_40456ffe-4d23-4e29-8a3d-845996885da8/graceful-termination/0.log" Jun 02 22:13:19.790979 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:19.790951 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25w52_3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d/kube-multus-additional-cni-plugins/0.log" Jun 02 22:13:19.808744 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:19.808716 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25w52_3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d/egress-router-binary-copy/0.log" Jun 02 22:13:19.826802 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:19.826779 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25w52_3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d/cni-plugins/0.log" Jun 02 22:13:19.848150 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:19.848127 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25w52_3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d/bond-cni-plugin/0.log" Jun 02 22:13:19.869028 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:19.869011 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25w52_3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d/routeoverride-cni/0.log" Jun 02 22:13:19.887853 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:19.887830 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25w52_3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d/whereabouts-cni-bincopy/0.log" Jun 02 22:13:19.905565 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:19.905548 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25w52_3b00cacd-fc6d-42ec-b64a-e3c9d2f60c6d/whereabouts-cni/0.log" Jun 02 22:13:20.214844 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:20.214817 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-d85n4_3e99f0a6-186b-4e51-ac27-e394f90e86b1/kube-multus/0.log" Jun 02 22:13:20.337455 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:20.337431 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p58r9_b4c54dfc-ecd1-466b-90db-eca67b4780c5/network-metrics-daemon/0.log" Jun 02 22:13:20.355498 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:20.355474 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p58r9_b4c54dfc-ecd1-466b-90db-eca67b4780c5/kube-rbac-proxy/0.log" Jun 02 22:13:21.691173 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:21.691146 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8npt_042b9156-fdec-4499-9bcc-d1a4f1f24aaf/ovn-controller/0.log" Jun 02 22:13:21.716701 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:21.716671 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8npt_042b9156-fdec-4499-9bcc-d1a4f1f24aaf/ovn-acl-logging/0.log" Jun 02 22:13:21.737263 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:21.737239 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8npt_042b9156-fdec-4499-9bcc-d1a4f1f24aaf/kube-rbac-proxy-node/0.log" Jun 02 22:13:21.759073 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:21.759054 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8npt_042b9156-fdec-4499-9bcc-d1a4f1f24aaf/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 22:13:21.772848 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:21.772833 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8npt_042b9156-fdec-4499-9bcc-d1a4f1f24aaf/northd/0.log" Jun 02 22:13:21.791945 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:21.791926 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8npt_042b9156-fdec-4499-9bcc-d1a4f1f24aaf/nbdb/0.log" Jun 02 22:13:21.810489 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:21.810466 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8npt_042b9156-fdec-4499-9bcc-d1a4f1f24aaf/sbdb/0.log" Jun 02 22:13:21.935836 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:21.935795 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8npt_042b9156-fdec-4499-9bcc-d1a4f1f24aaf/ovnkube-controller/0.log" Jun 02 22:13:23.025257 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:23.025231 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-vpfh6_301cea8a-d795-4207-8970-4dbec0729108/network-check-target-container/0.log" Jun 02 22:13:24.025985 ip-10-0-128-185 kubenswrapper[2562]: I0602 22:13:24.025956 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-s5hs9_c8c087a4-ad5e-40c4-8b41-4ce0bdbf762b/iptables-alerter/0.log"