Jul 07 11:24:59.260620 ip-10-0-139-127 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 11:24:59.260640 ip-10-0-139-127 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 11:24:59.260651 ip-10-0-139-127 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 11:24:59.261011 ip-10-0-139-127 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 11:25:09.448007 ip-10-0-139-127 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 11:25:09.448027 ip-10-0-139-127 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 81337601220741eabd1ed694d8fb8b2c -- Jul 07 11:27:05.274831 ip-10-0-139-127 systemd[1]: Starting Kubernetes Kubelet... Jul 07 11:27:05.706990 ip-10-0-139-127 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:27:05.706990 ip-10-0-139-127 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 11:27:05.706990 ip-10-0-139-127 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:27:05.706990 ip-10-0-139-127 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 11:27:05.706990 ip-10-0-139-127 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:27:05.708760 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.708673 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 11:27:05.713099 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713081 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:27:05.713099 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713099 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713103 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713107 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713111 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713114 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713117 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713120 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713124 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713126 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713130 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713133 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713136 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713138 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713141 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713143 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713148 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713151 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713160 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713164 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:27:05.713174 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713167 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713170 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713173 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713175 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713179 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713182 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713185 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713188 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713190 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713193 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713195 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713198 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713200 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713203 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713205 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713209 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713211 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713214 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713216 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713218 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:27:05.713646 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713221 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713223 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713226 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713229 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713231 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713234 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713238 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713241 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713244 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713246 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713248 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713251 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713254 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713256 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713259 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713262 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713264 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713267 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713270 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:27:05.714433 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713272 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713275 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713278 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713280 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713283 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713285 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713288 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713290 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713293 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713295 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713301 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713304 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713306 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713309 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713312 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713314 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713318 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713321 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713323 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713326 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:27:05.715105 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713329 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713331 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713335 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713337 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713339 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713342 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713345 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713348 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713351 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713353 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713356 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713358 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713361 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713363 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713366 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713368 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713371 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713373 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713377 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.713379 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:27:05.715610 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715343 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715354 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715358 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715361 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715365 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715368 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715371 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715374 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715377 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715379 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715382 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715384 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715387 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715390 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715393 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715395 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715398 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715401 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715403 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715411 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:27:05.716114 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715414 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715417 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715419 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715422 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715424 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715427 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715430 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715432 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715435 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715437 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715440 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715443 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715446 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715448 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715451 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715454 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715456 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715459 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715461 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:27:05.716642 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715465 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715469 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715473 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715476 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715478 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715481 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715484 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715487 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715489 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715492 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715495 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715498 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715501 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715504 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715507 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715510 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715513 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715515 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715518 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715520 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:27:05.717127 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715523 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715526 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715528 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715531 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715550 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715553 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715556 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715559 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715561 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715565 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715568 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715571 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715574 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715577 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715580 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715582 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715585 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715587 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715590 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715592 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:27:05.717637 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715595 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715598 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715601 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715603 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715608 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715613 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715617 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715620 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715623 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715626 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715629 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715632 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715635 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715638 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715641 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715645 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715648 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715651 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715654 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:27:05.718113 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.715656 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716053 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716059 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716062 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716065 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716068 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716071 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716074 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716076 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716078 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716081 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716084 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716086 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716089 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716091 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716094 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716097 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716102 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716105 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:27:05.718594 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716108 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716111 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716114 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716117 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716120 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716123 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716126 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716128 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716131 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716133 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716136 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716139 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716142 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716144 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716147 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716149 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716157 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716160 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716162 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:27:05.719057 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716165 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716167 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716170 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716172 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716175 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716177 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716180 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716182 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716185 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716187 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716190 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716193 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716196 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716198 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716201 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716204 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716207 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716209 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716212 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:27:05.719506 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716214 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716217 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716219 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716222 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716224 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716227 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716229 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716232 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716234 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716237 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716239 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716242 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716244 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716247 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716249 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716252 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716254 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716256 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716259 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716262 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:27:05.719982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716265 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716267 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716270 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716273 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716275 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716278 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716281 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716284 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716287 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716289 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716291 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716294 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716296 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716299 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716301 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716304 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716306 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716309 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716311 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716314 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:27:05.720463 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716318 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716321 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.716324 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716394 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716407 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716420 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716425 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716429 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716433 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716437 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716442 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716445 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716448 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716451 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716455 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716458 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716461 2578 flags.go:64] FLAG: --cgroup-root="" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716464 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716467 2578 flags.go:64] FLAG: --client-ca-file="" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716471 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716474 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716479 2578 flags.go:64] FLAG: --cluster-domain="" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716482 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716485 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 11:27:05.720982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716488 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716492 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716495 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716498 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716501 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716505 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716508 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716510 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716513 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716517 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716519 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716524 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716526 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716530 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716532 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716550 2578 flags.go:64] FLAG: --enable-server="true" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716553 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716558 2578 flags.go:64] FLAG: --event-burst="100" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716561 2578 flags.go:64] FLAG: --event-qps="50" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716564 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716568 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716571 2578 flags.go:64] FLAG: --eviction-hard="" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716575 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716578 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716581 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 11:27:05.721567 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716584 2578 flags.go:64] FLAG: --eviction-soft="" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716587 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716590 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716594 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716597 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716600 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716603 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716606 2578 flags.go:64] FLAG: --feature-gates="" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716609 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716612 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716619 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716622 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716625 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716629 2578 flags.go:64] FLAG: --help="false" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716631 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-139-127.ec2.internal" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716634 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716637 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716640 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716644 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716648 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716651 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716654 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716657 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 11:27:05.722155 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716662 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716665 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716668 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716671 2578 flags.go:64] FLAG: --kube-reserved="" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716674 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716676 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716679 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716682 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716685 2578 flags.go:64] FLAG: --lock-file="" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716688 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716691 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716694 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716699 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716703 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716706 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716709 2578 flags.go:64] FLAG: --logging-format="text" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716712 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716715 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716717 2578 flags.go:64] FLAG: --manifest-url="" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716722 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716726 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716729 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716734 2578 flags.go:64] FLAG: --max-pods="110" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716737 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716740 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 11:27:05.722771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716743 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716746 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716748 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716752 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716754 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716762 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716765 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716770 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716773 2578 flags.go:64] FLAG: --pod-cidr="" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716776 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716782 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716785 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716789 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716792 2578 flags.go:64] FLAG: --port="10250" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716795 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716798 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06df1bb511357664f" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716801 2578 flags.go:64] FLAG: --qos-reserved="" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716804 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716807 2578 flags.go:64] FLAG: --register-node="true" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716809 2578 flags.go:64] FLAG: --register-with-taints="" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716818 2578 flags.go:64] FLAG: --registry-burst="10" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716821 2578 flags.go:64] FLAG: --registry-qps="5" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716824 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716827 2578 flags.go:64] FLAG: --reserved-memory="" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716830 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 11:27:05.723386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716833 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716836 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716841 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716843 2578 flags.go:64] FLAG: --runonce="false" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716846 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716849 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716852 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716855 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716858 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716861 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716864 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716867 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716870 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716873 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716877 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716879 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716882 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716885 2578 flags.go:64] FLAG: --system-cgroups="" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716888 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716894 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716896 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716899 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716904 2578 flags.go:64] FLAG: --tls-min-version="" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716906 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716909 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 11:27:05.724020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716912 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716915 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716918 2578 flags.go:64] FLAG: --v="2" Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716922 2578 flags.go:64] FLAG: --version="false" Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716928 2578 flags.go:64] FLAG: --vmodule="" Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716931 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.716935 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717051 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717054 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717059 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717062 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717065 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717068 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717071 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717074 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717076 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717079 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717082 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717085 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717088 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717090 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:27:05.724691 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717097 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717100 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717102 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717105 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717108 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717111 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717113 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717117 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717121 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717124 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717127 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717129 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717132 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717135 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717137 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717140 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717143 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717146 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717148 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717151 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:27:05.725207 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717154 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717156 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717159 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717161 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717164 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717166 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717169 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717171 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717174 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717186 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717190 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717192 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717196 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717199 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717202 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717204 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717207 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717210 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717212 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717215 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:27:05.725709 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717217 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717220 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717223 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717225 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717228 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717230 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717233 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717235 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717237 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717241 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717244 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717246 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717249 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717251 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717254 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717256 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717259 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717261 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717264 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717266 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:27:05.726231 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717269 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717271 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717274 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717276 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717280 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717283 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717285 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717288 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717290 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717293 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717295 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717299 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717303 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717306 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717309 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717312 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717314 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717317 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717320 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:27:05.726739 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717324 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717328 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717331 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717335 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717338 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.717341 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.717880 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.724200 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.724217 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.724238 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.724245 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724309 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724316 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724320 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:27:05.727196 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724324 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724327 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724330 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724334 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724337 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724340 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724343 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724346 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724349 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724352 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724354 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724357 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724360 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724362 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724365 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724368 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724370 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724373 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724375 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724378 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:27:05.727567 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724380 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724383 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724387 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724390 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724393 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724396 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724399 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724402 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724406 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724409 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724412 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724415 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724418 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724421 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724424 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724427 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724429 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724432 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724435 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:27:05.728051 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724437 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724440 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724442 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724445 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724448 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724450 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724453 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724455 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724458 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724460 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724463 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724466 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724468 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724472 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724475 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724477 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724480 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724483 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724486 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:27:05.728518 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724488 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724491 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724494 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724497 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724499 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724502 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724504 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724507 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724509 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724512 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724514 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724517 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724519 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724522 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724525 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724527 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724529 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724532 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724550 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724554 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:27:05.729018 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724557 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724560 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724562 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724565 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724568 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724570 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724573 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724575 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724578 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724581 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724584 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724586 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724588 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724591 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724594 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724597 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724599 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:27:05.729508 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724602 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.724607 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724723 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724728 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724731 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724734 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724737 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724740 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724743 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724745 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724748 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724750 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724753 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724756 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724758 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:27:05.729938 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724761 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724763 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724766 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724769 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724771 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724774 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724777 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724780 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724782 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724785 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724788 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724790 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724793 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724796 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724799 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724802 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724805 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724807 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724810 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724812 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:27:05.730329 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724815 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724817 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724820 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724822 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724824 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724827 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724831 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724834 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724838 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724840 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724843 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724846 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724849 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724853 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724856 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724858 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724861 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724863 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724866 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724868 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:27:05.730840 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724871 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724873 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724876 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724878 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724881 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724883 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724886 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724888 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724891 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724893 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724896 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724898 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724901 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724903 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724906 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724908 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724911 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724913 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724916 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:27:05.731320 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724918 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724921 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724923 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724926 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724928 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724931 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724934 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724936 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724939 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724942 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724945 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724948 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724950 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724953 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724956 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724958 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724961 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724964 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724966 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:27:05.731782 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724969 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724971 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724974 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724977 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724979 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724982 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724984 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:05.724987 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.724991 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.725114 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.727025 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.728936 2578 server.go:1019] "Starting client certificate rotation" Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.729030 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.729066 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 11:27:05.732234 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.731265 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 11:27:05.732607 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.732438 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 11:27:05.743802 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.743776 2578 log.go:25] "Validated CRI v1 runtime API" Jul 07 11:27:05.745691 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.745674 2578 log.go:25] "Validated CRI v1 image API" Jul 07 11:27:05.746448 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.746432 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 11:27:05.751085 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.751058 2578 fs.go:135] Filesystem UUIDs: map[0fbe296b-6db3-4776-9538-b934490e6b02:/dev/nvme0n1p3 1031af0f-f7c9-4693-a11a-51d1b3752a9f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 07 11:27:05.751176 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.751083 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 11:27:05.756089 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.755969 2578 manager.go:216] Machine: {Timestamp:2026-07-07 11:27:05.754812978 +0000 UTC m=+0.359149809 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102356 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e662e98499ae9b97577a32909ff26 SystemUUID:ec2e662e-9849-9ae9-b975-77a32909ff26 BootID:81337601-2207-41ea-bd1e-d694d8fb8b2c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:30:ab:a2:0b:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:30:ab:a2:0b:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:06:e6:d5:a6:37 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 11:27:05.756089 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.756076 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 11:27:05.756257 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.756183 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 11:27:05.759506 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.759478 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 11:27:05.759674 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.759513 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-127.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 11:27:05.760139 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.760128 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 11:27:05.760173 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.760141 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 11:27:05.760173 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.760149 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 11:27:05.760649 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.760635 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 11:27:05.760709 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.760695 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 11:27:05.761928 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.761917 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 11:27:05.764390 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.764378 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 07 11:27:05.764507 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.764498 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 11:27:05.764553 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.764528 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 07 11:27:05.764591 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.764554 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 11:27:05.764591 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.764568 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 11:27:05.764591 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.764578 2578 kubelet.go:398] "Adding apiserver pod source" Jul 07 11:27:05.764591 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.764587 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 11:27:05.764963 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.764951 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 11:27:05.765007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.764969 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 11:27:05.766916 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.766898 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 11:27:05.768589 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.768572 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 11:27:05.768676 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.768611 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 11:27:05.770221 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.770207 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 11:27:05.770303 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.770231 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 11:27:05.770303 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.770240 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 11:27:05.770303 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.770249 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 11:27:05.770303 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.770261 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 11:27:05.770303 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.770271 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 11:27:05.770303 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.770285 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 11:27:05.770303 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.770294 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 11:27:05.770303 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.770304 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 11:27:05.770531 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.770313 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 11:27:05.770531 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.770325 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 11:27:05.770531 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.770335 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 11:27:05.772035 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.772018 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 11:27:05.772035 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.772033 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 11:27:05.774609 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.774573 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-127.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 11:27:05.774792 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.774748 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-127.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 11:27:05.774967 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.774942 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 11:27:05.775893 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.775875 2578 server.go:1268] "Started kubelet" Jul 07 11:27:05.776008 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.775977 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 11:27:05.776177 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.775981 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 11:27:05.776246 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.776202 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 11:27:05.776888 ip-10-0-139-127 systemd[1]: Started Kubernetes Kubelet. Jul 07 11:27:05.777601 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.777586 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 11:27:05.779969 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.779953 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 07 11:27:05.781670 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.781649 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 11:27:05.783740 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.783726 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 11:27:05.783817 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.783740 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 11:27:05.783879 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.783835 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:05.784392 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.784377 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 11:27:05.784392 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.784390 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 11:27:05.784520 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.784406 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 11:27:05.784578 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.784562 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 07 11:27:05.784578 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.784573 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 07 11:27:05.784683 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.784663 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 11:27:05.784683 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.784675 2578 factory.go:55] Registering systemd factory Jul 07 11:27:05.784683 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.784683 2578 factory.go:223] Registration of the systemd container factory successfully Jul 07 11:27:05.784883 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.784871 2578 factory.go:153] Registering CRI-O factory Jul 07 11:27:05.784883 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.784882 2578 factory.go:223] Registration of the crio container factory successfully Jul 07 11:27:05.784973 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.784899 2578 factory.go:103] Registering Raw factory Jul 07 11:27:05.784973 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.784908 2578 manager.go:1196] Started watching for new ooms in manager Jul 07 11:27:05.785447 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.785432 2578 manager.go:349] Starting recovery of all containers Jul 07 11:27:05.787020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.786994 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xwp4t" Jul 07 11:27:05.787097 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.785242 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-127.ec2.internal.18bffe56b268548e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-127.ec2.internal,UID:ip-10-0-139-127.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-127.ec2.internal,},FirstTimestamp:2026-07-07 11:27:05.775830158 +0000 UTC m=+0.380166992,LastTimestamp:2026-07-07 11:27:05.775830158 +0000 UTC m=+0.380166992,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-127.ec2.internal,}" Jul 07 11:27:05.787194 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.787171 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 11:27:05.792107 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.792073 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 11:27:05.794826 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.794672 2578 manager.go:354] Recovery completed Jul 07 11:27:05.794995 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.794975 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xwp4t" Jul 07 11:27:05.797709 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.797669 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-127.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 11:27:05.800531 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.800519 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:27:05.802315 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.802300 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:27:05.802385 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.802327 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:27:05.802385 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.802337 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:27:05.802943 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.802931 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 11:27:05.802943 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.802942 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 11:27:05.803025 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.802960 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 07 11:27:05.803025 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.802944 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:27:05.804100 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.804089 2578 policy_none.go:49] "None policy: Start" Jul 07 11:27:05.804137 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.804104 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 11:27:05.804137 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.804114 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 11:27:05.805214 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.805203 2578 policy_none.go:47] "Start" Jul 07 11:27:05.805931 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.805915 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:27:05.806007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.805944 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:27:05.806007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.805959 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:27:05.846232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.846204 2578 manager.go:337] "Starting Device Plugin manager" Jul 07 11:27:05.846343 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.846242 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 11:27:05.846343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.846252 2578 server.go:87] "Starting device plugin registration server" Jul 07 11:27:05.846554 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.846526 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 11:27:05.846612 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.846558 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 11:27:05.846670 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.846643 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 11:27:05.846763 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.846741 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 11:27:05.846763 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.846751 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 11:27:05.847327 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.847308 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 11:27:05.847398 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.847359 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:05.931871 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.931840 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 11:27:05.932813 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.931882 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 11:27:05.932813 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.931909 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 11:27:05.932813 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.931948 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 11:27:05.934447 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.934424 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:27:05.946706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.946679 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:27:05.947549 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.947522 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:27:05.947612 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.947563 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:27:05.947612 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.947575 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:27:05.947612 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.947598 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-127.ec2.internal" Jul 07 11:27:05.953696 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:05.953680 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-127.ec2.internal" Jul 07 11:27:05.953761 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.953702 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-127.ec2.internal\": node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:05.967849 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:05.967796 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:06.032658 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.032625 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"] Jul 07 11:27:06.032771 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.032701 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:27:06.033673 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.033659 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:27:06.033738 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.033686 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:27:06.033738 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.033699 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:27:06.034821 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.034807 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:27:06.034929 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.034913 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.034982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.034948 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:27:06.035597 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.035582 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:27:06.035597 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.035582 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:27:06.035702 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.035609 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:27:06.035702 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.035615 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:27:06.035702 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.035630 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:27:06.035702 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.035618 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:27:06.036681 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.036667 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.036742 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.036695 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:27:06.037385 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.037365 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:27:06.037483 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.037399 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:27:06.037483 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.037413 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:27:06.058290 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:06.058263 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-127.ec2.internal\" not found" node="ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.062566 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:06.062550 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-127.ec2.internal\" not found" node="ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.068584 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:06.068568 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:06.085688 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.085664 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a7d7affa3453ec77ef6377c3782056c0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"a7d7affa3453ec77ef6377c3782056c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.085782 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.085693 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7d7affa3453ec77ef6377c3782056c0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"a7d7affa3453ec77ef6377c3782056c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.085782 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.085711 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85d1b63fc295b631cf34855fb24fcee0-config\") pod \"kube-apiserver-proxy-ip-10-0-139-127.ec2.internal\" (UID: \"85d1b63fc295b631cf34855fb24fcee0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.169159 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:06.169131 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:06.186435 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.186409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85d1b63fc295b631cf34855fb24fcee0-config\") pod \"kube-apiserver-proxy-ip-10-0-139-127.ec2.internal\" (UID: \"85d1b63fc295b631cf34855fb24fcee0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.186517 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.186441 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a7d7affa3453ec77ef6377c3782056c0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"a7d7affa3453ec77ef6377c3782056c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.186517 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.186457 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7d7affa3453ec77ef6377c3782056c0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"a7d7affa3453ec77ef6377c3782056c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.186517 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.186508 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a7d7affa3453ec77ef6377c3782056c0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"a7d7affa3453ec77ef6377c3782056c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.186682 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.186512 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85d1b63fc295b631cf34855fb24fcee0-config\") pod \"kube-apiserver-proxy-ip-10-0-139-127.ec2.internal\" (UID: \"85d1b63fc295b631cf34855fb24fcee0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.186682 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.186580 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7d7affa3453ec77ef6377c3782056c0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"a7d7affa3453ec77ef6377c3782056c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.270049 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:06.269955 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:06.361588 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.361533 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.365190 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.365095 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" Jul 07 11:27:06.370373 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:06.370355 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:06.470937 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:06.470897 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:06.571531 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:06.571453 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:06.672062 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:06.672029 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:06.727591 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.727565 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 11:27:06.728106 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.727743 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 11:27:06.773129 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:06.773100 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:06.797608 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.797573 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 11:22:05 +0000 UTC" deadline="2028-01-17 22:35:23.335237629 +0000 UTC" Jul 07 11:27:06.797608 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.797605 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13427h8m16.537635673s" Jul 07 11:27:06.873967 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:06.873903 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:06.890007 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:06.889971 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d1b63fc295b631cf34855fb24fcee0.slice/crio-ed5f2bd5e9ed711c95aa7f747656214652d24cb08df77c046001c5ef84cf3824 WatchSource:0}: Error finding container ed5f2bd5e9ed711c95aa7f747656214652d24cb08df77c046001c5ef84cf3824: Status 404 returned error can't find the container with id ed5f2bd5e9ed711c95aa7f747656214652d24cb08df77c046001c5ef84cf3824 Jul 07 11:27:06.890411 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:06.890385 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7d7affa3453ec77ef6377c3782056c0.slice/crio-5630696792bf6eb55a7cc5ccd71d730da594647f7eac7a5eb01c2dedc5866961 WatchSource:0}: Error finding container 5630696792bf6eb55a7cc5ccd71d730da594647f7eac7a5eb01c2dedc5866961: Status 404 returned error can't find the container with id 5630696792bf6eb55a7cc5ccd71d730da594647f7eac7a5eb01c2dedc5866961 Jul 07 11:27:06.894735 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.894720 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:27:06.933629 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.933603 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:27:06.935643 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.935599 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" event={"ID":"85d1b63fc295b631cf34855fb24fcee0","Type":"ContainerStarted","Data":"ed5f2bd5e9ed711c95aa7f747656214652d24cb08df77c046001c5ef84cf3824"} Jul 07 11:27:06.936663 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:06.936644 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" event={"ID":"a7d7affa3453ec77ef6377c3782056c0","Type":"ContainerStarted","Data":"5630696792bf6eb55a7cc5ccd71d730da594647f7eac7a5eb01c2dedc5866961"} Jul 07 11:27:06.974827 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:06.974778 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:07.075339 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:07.075304 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found" Jul 07 11:27:07.118449 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.118425 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:27:07.183894 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.183871 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" Jul 07 11:27:07.194235 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.194212 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 11:27:07.195943 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.195928 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" Jul 07 11:27:07.206115 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.206098 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 11:27:07.373991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.373958 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:27:07.765431 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.765391 2578 apiserver.go:52] "Watching apiserver" Jul 07 11:27:07.772981 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.772954 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 11:27:07.774000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.773972 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-hvrdw","openshift-ovn-kubernetes/ovnkube-node-7r54t","kube-system/konnectivity-agent-v9wwc","kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc","openshift-dns/node-resolver-nqrsk","openshift-image-registry/node-ca-rnzn8","openshift-multus/multus-additional-cni-plugins-5kl2h","openshift-multus/multus-vbmng","openshift-multus/network-metrics-daemon-9vm97","kube-system/global-pull-secret-syncer-6kjkj","openshift-cluster-node-tuning-operator/tuned-74htd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal","openshift-network-diagnostics/network-check-target-zn5pv"] Jul 07 11:27:07.775679 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.775641 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rnzn8" Jul 07 11:27:07.776772 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.776751 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:07.776864 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:07.776816 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:07.777837 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.777813 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hvrdw" Jul 07 11:27:07.778280 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.778261 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bszwz\"" Jul 07 11:27:07.778280 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.778259 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 11:27:07.778421 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.778299 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 11:27:07.778834 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.778666 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 11:27:07.779020 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.779004 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.779830 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.779812 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:27:07.780047 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.780032 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 11:27:07.780390 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.780375 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 11:27:07.780595 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.780579 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vdczj\"" Jul 07 11:27:07.781266 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.781247 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 11:27:07.781350 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.781285 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 11:27:07.781527 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.781510 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 11:27:07.781629 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.781525 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 11:27:07.782038 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.782020 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-twsnl\"" Jul 07 11:27:07.783079 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.783062 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:07.783167 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:07.783141 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:07.784478 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.784440 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nqrsk" Jul 07 11:27:07.784725 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.784442 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 11:27:07.785855 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.785837 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.786565 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.786529 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mpjj2\"" Jul 07 11:27:07.786660 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.786610 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 11:27:07.786718 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.786530 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 11:27:07.787414 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.787394 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:07.787888 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.787871 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 11:27:07.788101 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.788084 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 11:27:07.788170 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.788121 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 11:27:07.788170 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.788133 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bnsx4\"" Jul 07 11:27:07.788406 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.788387 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v9wwc" Jul 07 11:27:07.789496 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.789478 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 11:27:07.789604 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.789517 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 11:27:07.789673 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.789626 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:07.789729 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:07.789684 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:07.790074 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.790054 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9sg47\"" Jul 07 11:27:07.790395 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.790373 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 11:27:07.790795 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.790758 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 11:27:07.791899 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.791878 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8plfh\"" Jul 07 11:27:07.795409 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.793604 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.795409 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795377 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d38329b0-4add-42cf-ba11-ff2643797fa1-host-slash\") pod \"iptables-alerter-hvrdw\" (UID: \"d38329b0-4add-42cf-ba11-ff2643797fa1\") " pod="openshift-network-operator/iptables-alerter-hvrdw" Jul 07 11:27:07.795576 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795432 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-multus-cni-dir\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.795576 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795468 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-run-k8s-cni-cncf-io\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.795576 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795499 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-run-netns\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.795735 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795576 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-sys-fs\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.795735 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795618 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-system-cni-dir\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.795735 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-var-lib-cni-bin\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.795735 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-var-lib-kubelet\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.795735 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ba1d1827-3e47-4686-8173-7c4619849f84-hosts-file\") pod \"node-resolver-nqrsk\" (UID: \"ba1d1827-3e47-4686-8173-7c4619849f84\") " pod="openshift-dns/node-resolver-nqrsk" Jul 07 11:27:07.795991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795764 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba1d1827-3e47-4686-8173-7c4619849f84-tmp-dir\") pod \"node-resolver-nqrsk\" (UID: \"ba1d1827-3e47-4686-8173-7c4619849f84\") " pod="openshift-dns/node-resolver-nqrsk" Jul 07 11:27:07.795991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795798 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt72t\" (UniqueName: \"kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t\") pod \"network-check-target-zn5pv\" (UID: \"8341686d-bc8f-4896-96cf-558ba701f0e0\") " pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:07.795991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795829 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbrrf\" (UniqueName: \"kubernetes.io/projected/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-kube-api-access-mbrrf\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.795991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795858 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg8zw\" (UniqueName: \"kubernetes.io/projected/ba1d1827-3e47-4686-8173-7c4619849f84-kube-api-access-mg8zw\") pod \"node-resolver-nqrsk\" (UID: \"ba1d1827-3e47-4686-8173-7c4619849f84\") " pod="openshift-dns/node-resolver-nqrsk" Jul 07 11:27:07.795991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795893 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8319a2e-d262-43be-b188-7d50ac4598be-host\") pod \"node-ca-rnzn8\" (UID: \"a8319a2e-d262-43be-b188-7d50ac4598be\") " pod="openshift-image-registry/node-ca-rnzn8" Jul 07 11:27:07.795991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8g5v\" (UniqueName: \"kubernetes.io/projected/a8319a2e-d262-43be-b188-7d50ac4598be-kube-api-access-t8g5v\") pod \"node-ca-rnzn8\" (UID: \"a8319a2e-d262-43be-b188-7d50ac4598be\") " pod="openshift-image-registry/node-ca-rnzn8" Jul 07 11:27:07.795991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.795986 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-cnibin\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-run-multus-certs\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-os-release\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796147 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-registration-dir\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796166 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jcrvv\"" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796177 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-device-dir\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796186 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-etc-selinux\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796234 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p5kb\" (UniqueName: \"kubernetes.io/projected/d38329b0-4add-42cf-ba11-ff2643797fa1-kube-api-access-2p5kb\") pod \"iptables-alerter-hvrdw\" (UID: \"d38329b0-4add-42cf-ba11-ff2643797fa1\") " pod="openshift-network-operator/iptables-alerter-hvrdw" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796266 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-var-lib-cni-multus\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796288 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796297 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d38329b0-4add-42cf-ba11-ff2643797fa1-iptables-alerter-script\") pod \"iptables-alerter-hvrdw\" (UID: \"d38329b0-4add-42cf-ba11-ff2643797fa1\") " pod="openshift-network-operator/iptables-alerter-hvrdw" Jul 07 11:27:07.796325 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796331 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-multus-socket-dir-parent\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.796989 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796415 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-multus-conf-dir\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.796989 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-multus-daemon-config\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.796989 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796509 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:07.796989 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kt4w\" (UniqueName: \"kubernetes.io/projected/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-kube-api-access-4kt4w\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:07.796989 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796592 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8319a2e-d262-43be-b188-7d50ac4598be-serviceca\") pod \"node-ca-rnzn8\" (UID: \"a8319a2e-d262-43be-b188-7d50ac4598be\") " pod="openshift-image-registry/node-ca-rnzn8" Jul 07 11:27:07.796989 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-cni-binary-copy\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.796989 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796694 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-hostroot\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.796989 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-etc-kubernetes\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.796989 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-socket-dir\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.796989 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.796797 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttrhj\" (UniqueName: \"kubernetes.io/projected/f94c027e-3db7-4431-9687-dcb104f46ac6-kube-api-access-ttrhj\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.797844 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.797825 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.799504 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.799483 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 11:27:07.800393 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.799950 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 11:27:07.800393 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.799991 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 11:27:07.800502 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.800473 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 11:27:07.800572 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.800505 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cc9q6\"" Jul 07 11:27:07.800625 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.800615 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 11:27:07.800963 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.800744 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 11:27:07.800963 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.800801 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 11:27:07.818833 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.818807 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zwl8x" Jul 07 11:27:07.826751 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.826707 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zwl8x" Jul 07 11:27:07.868983 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.868958 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:27:07.886002 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.885979 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 11:27:07.897447 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-etc-openvswitch\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.897576 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897453 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9d6ea786-e03d-4f74-a080-914765d20aef-agent-certs\") pod \"konnectivity-agent-v9wwc\" (UID: \"9d6ea786-e03d-4f74-a080-914765d20aef\") " pod="kube-system/konnectivity-agent-v9wwc" Jul 07 11:27:07.897576 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-multus-cni-dir\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.897576 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897547 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-sys-fs\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.897679 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897575 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-multus-cni-dir\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.897679 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897615 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-sys-fs\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.897679 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897577 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-kubernetes\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.897679 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897641 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-run-openvswitch\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.897679 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897657 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-cni-bin\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.897858 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-var-lib-cni-bin\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.897858 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897696 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ba1d1827-3e47-4686-8173-7c4619849f84-hosts-file\") pod \"node-resolver-nqrsk\" (UID: \"ba1d1827-3e47-4686-8173-7c4619849f84\") " pod="openshift-dns/node-resolver-nqrsk" Jul 07 11:27:07.897858 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897712 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/674e54c4-6cb6-4fbc-ba4e-743e240039a7-os-release\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:07.897858 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-run-ovn\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.897858 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-cni-netd\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.897858 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897788 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ba1d1827-3e47-4686-8173-7c4619849f84-hosts-file\") pod \"node-resolver-nqrsk\" (UID: \"ba1d1827-3e47-4686-8173-7c4619849f84\") " pod="openshift-dns/node-resolver-nqrsk" Jul 07 11:27:07.897858 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897796 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-var-lib-cni-bin\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.897858 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897822 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.897858 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897844 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt72t\" (UniqueName: \"kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t\") pod \"network-check-target-zn5pv\" (UID: \"8341686d-bc8f-4896-96cf-558ba701f0e0\") " pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:07.897858 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897861 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/674e54c4-6cb6-4fbc-ba4e-743e240039a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9d6ea786-e03d-4f74-a080-914765d20aef-konnectivity-ca\") pod \"konnectivity-agent-v9wwc\" (UID: \"9d6ea786-e03d-4f74-a080-914765d20aef\") " pod="kube-system/konnectivity-agent-v9wwc" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897893 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8g5v\" (UniqueName: \"kubernetes.io/projected/a8319a2e-d262-43be-b188-7d50ac4598be-kube-api-access-t8g5v\") pod \"node-ca-rnzn8\" (UID: \"a8319a2e-d262-43be-b188-7d50ac4598be\") " pod="openshift-image-registry/node-ca-rnzn8" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-cnibin\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897923 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-run-multus-certs\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897937 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/674e54c4-6cb6-4fbc-ba4e-743e240039a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897963 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897979 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-cnibin\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.897989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-registration-dir\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898009 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-run-multus-certs\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898033 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-device-dir\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898067 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-device-dir\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898071 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-node-log\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898073 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-registration-dir\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898106 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-var-lib-cni-multus\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898122 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-var-lib-cni-multus\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.898320 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898128 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d38329b0-4add-42cf-ba11-ff2643797fa1-iptables-alerter-script\") pod \"iptables-alerter-hvrdw\" (UID: \"d38329b0-4add-42cf-ba11-ff2643797fa1\") " pod="openshift-network-operator/iptables-alerter-hvrdw" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898149 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/674e54c4-6cb6-4fbc-ba4e-743e240039a7-cnibin\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898166 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/674e54c4-6cb6-4fbc-ba4e-743e240039a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898202 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-log-socket\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-multus-conf-dir\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898282 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-multus-conf-dir\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898311 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kt4w\" (UniqueName: \"kubernetes.io/projected/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-kube-api-access-4kt4w\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898341 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-sysctl-d\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:07.898376 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:07.898466 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs podName:e825e10d-32d1-41d4-aa12-1d1f1dbbb429 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:08.398439091 +0000 UTC m=+3.002775915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs") pod "network-metrics-daemon-9vm97" (UID: "e825e10d-32d1-41d4-aa12-1d1f1dbbb429") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-sysctl-conf\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898550 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8319a2e-d262-43be-b188-7d50ac4598be-serviceca\") pod \"node-ca-rnzn8\" (UID: \"a8319a2e-d262-43be-b188-7d50ac4598be\") " pod="openshift-image-registry/node-ca-rnzn8" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898578 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttrhj\" (UniqueName: \"kubernetes.io/projected/f94c027e-3db7-4431-9687-dcb104f46ac6-kube-api-access-ttrhj\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898590 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d38329b0-4add-42cf-ba11-ff2643797fa1-iptables-alerter-script\") pod \"iptables-alerter-hvrdw\" (UID: \"d38329b0-4add-42cf-ba11-ff2643797fa1\") " pod="openshift-network-operator/iptables-alerter-hvrdw" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d38329b0-4add-42cf-ba11-ff2643797fa1-host-slash\") pod \"iptables-alerter-hvrdw\" (UID: \"d38329b0-4add-42cf-ba11-ff2643797fa1\") " pod="openshift-network-operator/iptables-alerter-hvrdw" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-run\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.899000 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898644 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-lib-modules\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898654 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d38329b0-4add-42cf-ba11-ff2643797fa1-host-slash\") pod \"iptables-alerter-hvrdw\" (UID: \"d38329b0-4add-42cf-ba11-ff2643797fa1\") " pod="openshift-network-operator/iptables-alerter-hvrdw" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-host\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-run-netns\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898741 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-var-lib-openvswitch\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-run-k8s-cni-cncf-io\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-run-netns\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898811 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-run-k8s-cni-cncf-io\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-kubelet\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898868 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-run-netns\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898893 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-ovnkube-config\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0eafca62-9210-42e4-8af3-15faf4201332-kubelet-config\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898942 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898959 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-system-cni-dir\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898976 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-var-lib-kubelet\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.898991 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba1d1827-3e47-4686-8173-7c4619849f84-tmp-dir\") pod \"node-resolver-nqrsk\" (UID: \"ba1d1827-3e47-4686-8173-7c4619849f84\") " pod="openshift-dns/node-resolver-nqrsk" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-var-lib-kubelet\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.899680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899022 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8319a2e-d262-43be-b188-7d50ac4598be-serviceca\") pod \"node-ca-rnzn8\" (UID: \"a8319a2e-d262-43be-b188-7d50ac4598be\") " pod="openshift-image-registry/node-ca-rnzn8" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899026 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-system-cni-dir\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899044 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-host-var-lib-kubelet\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-env-overrides\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899128 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-ovn-node-metrics-cert\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899162 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbrrf\" (UniqueName: \"kubernetes.io/projected/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-kube-api-access-mbrrf\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899188 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg8zw\" (UniqueName: \"kubernetes.io/projected/ba1d1827-3e47-4686-8173-7c4619849f84-kube-api-access-mg8zw\") pod \"node-resolver-nqrsk\" (UID: \"ba1d1827-3e47-4686-8173-7c4619849f84\") " pod="openshift-dns/node-resolver-nqrsk" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899203 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-sys\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899218 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj4z9\" (UniqueName: \"kubernetes.io/projected/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-kube-api-access-cj4z9\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899235 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-systemd-units\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899263 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba1d1827-3e47-4686-8173-7c4619849f84-tmp-dir\") pod \"node-resolver-nqrsk\" (UID: \"ba1d1827-3e47-4686-8173-7c4619849f84\") " pod="openshift-dns/node-resolver-nqrsk" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899327 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-ovnkube-script-lib\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899374 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8319a2e-d262-43be-b188-7d50ac4598be-host\") pod \"node-ca-rnzn8\" (UID: \"a8319a2e-d262-43be-b188-7d50ac4598be\") " pod="openshift-image-registry/node-ca-rnzn8" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899398 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-sysconfig\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899443 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8319a2e-d262-43be-b188-7d50ac4598be-host\") pod \"node-ca-rnzn8\" (UID: \"a8319a2e-d262-43be-b188-7d50ac4598be\") " pod="openshift-image-registry/node-ca-rnzn8" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lqg8\" (UniqueName: \"kubernetes.io/projected/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-kube-api-access-6lqg8\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899510 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0eafca62-9210-42e4-8af3-15faf4201332-dbus\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:07.900473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899554 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-os-release\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899595 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-etc-selinux\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899609 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-os-release\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899626 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p5kb\" (UniqueName: \"kubernetes.io/projected/d38329b0-4add-42cf-ba11-ff2643797fa1-kube-api-access-2p5kb\") pod \"iptables-alerter-hvrdw\" (UID: \"d38329b0-4add-42cf-ba11-ff2643797fa1\") " pod="openshift-network-operator/iptables-alerter-hvrdw" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899653 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/674e54c4-6cb6-4fbc-ba4e-743e240039a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899677 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-modprobe-d\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899698 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-systemd\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899698 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-etc-selinux\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899740 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-tuned\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-multus-socket-dir-parent\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899790 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-multus-daemon-config\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899820 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/674e54c4-6cb6-4fbc-ba4e-743e240039a7-system-cni-dir\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899852 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-multus-socket-dir-parent\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900197 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-multus-daemon-config\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.899848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjtdk\" (UniqueName: \"kubernetes.io/projected/674e54c4-6cb6-4fbc-ba4e-743e240039a7-kube-api-access-wjtdk\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900306 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-tmp\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-run-systemd\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.901007 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-run-ovn-kubernetes\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.901470 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900410 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-cni-binary-copy\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.901470 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900434 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-hostroot\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.901470 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900478 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-etc-kubernetes\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.901470 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-socket-dir\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.901470 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900480 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-hostroot\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.901470 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900562 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-slash\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:07.901470 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900572 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-etc-kubernetes\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.901470 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900647 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f94c027e-3db7-4431-9687-dcb104f46ac6-socket-dir\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:07.901470 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.900901 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-cni-binary-copy\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.907682 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:07.907665 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:27:07.907682 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:07.907683 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:27:07.907796 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:07.907692 2578 projected.go:196] Error preparing data for projected volume kube-api-access-bt72t for pod openshift-network-diagnostics/network-check-target-zn5pv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:07.907796 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:07.907747 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t podName:8341686d-bc8f-4896-96cf-558ba701f0e0 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:08.407731066 +0000 UTC m=+3.012067885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bt72t" (UniqueName: "kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t") pod "network-check-target-zn5pv" (UID: "8341686d-bc8f-4896-96cf-558ba701f0e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:07.908812 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.908790 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 11:27:07.912773 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.912739 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8g5v\" (UniqueName: \"kubernetes.io/projected/a8319a2e-d262-43be-b188-7d50ac4598be-kube-api-access-t8g5v\") pod \"node-ca-rnzn8\" (UID: \"a8319a2e-d262-43be-b188-7d50ac4598be\") " pod="openshift-image-registry/node-ca-rnzn8" Jul 07 11:27:07.914167 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.914137 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p5kb\" (UniqueName: \"kubernetes.io/projected/d38329b0-4add-42cf-ba11-ff2643797fa1-kube-api-access-2p5kb\") pod \"iptables-alerter-hvrdw\" (UID: \"d38329b0-4add-42cf-ba11-ff2643797fa1\") " pod="openshift-network-operator/iptables-alerter-hvrdw" Jul 07 11:27:07.914300 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.914282 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg8zw\" (UniqueName: \"kubernetes.io/projected/ba1d1827-3e47-4686-8173-7c4619849f84-kube-api-access-mg8zw\") pod \"node-resolver-nqrsk\" (UID: \"ba1d1827-3e47-4686-8173-7c4619849f84\") " pod="openshift-dns/node-resolver-nqrsk" Jul 07 11:27:07.914382 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.914362 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kt4w\" (UniqueName: \"kubernetes.io/projected/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-kube-api-access-4kt4w\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:07.915767 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.915743 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbrrf\" (UniqueName: \"kubernetes.io/projected/6d9acbc8-5af1-4514-bc47-b8c50a06d5a3-kube-api-access-mbrrf\") pod \"multus-vbmng\" (UID: \"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3\") " pod="openshift-multus/multus-vbmng" Jul 07 11:27:07.915933 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:07.915909 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttrhj\" (UniqueName: \"kubernetes.io/projected/f94c027e-3db7-4431-9687-dcb104f46ac6-kube-api-access-ttrhj\") pod \"aws-ebs-csi-driver-node-nrckc\" (UID: \"f94c027e-3db7-4431-9687-dcb104f46ac6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:08.001017 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.000982 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-sys\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.001170 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001026 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4z9\" (UniqueName: \"kubernetes.io/projected/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-kube-api-access-cj4z9\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.001170 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001052 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-systemd-units\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.001170 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001100 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-systemd-units\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.001170 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001121 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-sys\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.001170 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001137 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-ovnkube-script-lib\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.001398 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001178 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-sysconfig\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.001398 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001205 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lqg8\" (UniqueName: \"kubernetes.io/projected/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-kube-api-access-6lqg8\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.001398 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001228 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0eafca62-9210-42e4-8af3-15faf4201332-dbus\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:08.001398 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/674e54c4-6cb6-4fbc-ba4e-743e240039a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.001398 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001308 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-sysconfig\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.001398 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001343 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-modprobe-d\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.001398 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001365 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-systemd\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.001398 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001366 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0eafca62-9210-42e4-8af3-15faf4201332-dbus\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:08.001398 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-tuned\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001413 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/674e54c4-6cb6-4fbc-ba4e-743e240039a7-system-cni-dir\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001439 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjtdk\" (UniqueName: \"kubernetes.io/projected/674e54c4-6cb6-4fbc-ba4e-743e240039a7-kube-api-access-wjtdk\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001465 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-tmp\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001491 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-run-systemd\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001516 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-run-ovn-kubernetes\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-slash\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001586 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/674e54c4-6cb6-4fbc-ba4e-743e240039a7-system-cni-dir\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001591 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-etc-openvswitch\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001623 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9d6ea786-e03d-4f74-a080-914765d20aef-agent-certs\") pod \"konnectivity-agent-v9wwc\" (UID: \"9d6ea786-e03d-4f74-a080-914765d20aef\") " pod="kube-system/konnectivity-agent-v9wwc" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001633 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-etc-openvswitch\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001644 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-kubernetes\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001662 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-run-openvswitch\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001683 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-cni-bin\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.001706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001705 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/674e54c4-6cb6-4fbc-ba4e-743e240039a7-os-release\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-run-ovn\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-cni-netd\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001737 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-ovnkube-script-lib\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001752 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001803 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/674e54c4-6cb6-4fbc-ba4e-743e240039a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001811 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-cni-bin\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001827 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-kubernetes\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001871 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-run-ovn-kubernetes\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001882 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/674e54c4-6cb6-4fbc-ba4e-743e240039a7-os-release\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001888 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-slash\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001887 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/674e54c4-6cb6-4fbc-ba4e-743e240039a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001861 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-cni-netd\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001942 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-systemd\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001945 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-run-openvswitch\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001954 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-run-systemd\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.001984 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-run-ovn\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.002343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002030 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9d6ea786-e03d-4f74-a080-914765d20aef-konnectivity-ca\") pod \"konnectivity-agent-v9wwc\" (UID: \"9d6ea786-e03d-4f74-a080-914765d20aef\") " pod="kube-system/konnectivity-agent-v9wwc" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/674e54c4-6cb6-4fbc-ba4e-743e240039a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002112 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-node-log\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002120 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-modprobe-d\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002127 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/674e54c4-6cb6-4fbc-ba4e-743e240039a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/674e54c4-6cb6-4fbc-ba4e-743e240039a7-cnibin\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002191 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/674e54c4-6cb6-4fbc-ba4e-743e240039a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002215 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-log-socket\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002279 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-node-log\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002283 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-sysctl-d\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-sysctl-conf\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002348 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/674e54c4-6cb6-4fbc-ba4e-743e240039a7-cnibin\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-run\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002382 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-lib-modules\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002395 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-run\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002396 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-sysctl-d\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002407 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-host\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.003232 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-run-netns\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002480 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-host\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-var-lib-openvswitch\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002513 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-kubelet\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002552 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-ovnkube-config\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002561 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-run-netns\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002577 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0eafca62-9210-42e4-8af3-15faf4201332-kubelet-config\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002592 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-lib-modules\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002610 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-kubelet\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002631 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-var-lib-openvswitch\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-var-lib-kubelet\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-env-overrides\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002679 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:08.002683 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002694 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-ovn-node-metrics-cert\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:08.002742 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret podName:0eafca62-9210-42e4-8af3-15faf4201332 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:08.502723405 +0000 UTC m=+3.107060231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret") pod "global-pull-secret-syncer-6kjkj" (UID: "0eafca62-9210-42e4-8af3-15faf4201332") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:08.004040 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002746 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/674e54c4-6cb6-4fbc-ba4e-743e240039a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.004839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002821 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-var-lib-kubelet\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.004839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002827 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9d6ea786-e03d-4f74-a080-914765d20aef-konnectivity-ca\") pod \"konnectivity-agent-v9wwc\" (UID: \"9d6ea786-e03d-4f74-a080-914765d20aef\") " pod="kube-system/konnectivity-agent-v9wwc" Jul 07 11:27:08.004839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002885 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-log-socket\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.002917 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0eafca62-9210-42e4-8af3-15faf4201332-kubelet-config\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:08.004839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.003109 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-sysctl-conf\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.004839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.003182 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-ovnkube-config\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.003196 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-env-overrides\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.004839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.003295 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/674e54c4-6cb6-4fbc-ba4e-743e240039a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.004839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.004398 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-tmp\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.004839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.004650 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9d6ea786-e03d-4f74-a080-914765d20aef-agent-certs\") pod \"konnectivity-agent-v9wwc\" (UID: \"9d6ea786-e03d-4f74-a080-914765d20aef\") " pod="kube-system/konnectivity-agent-v9wwc" Jul 07 11:27:08.004839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.004804 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-etc-tuned\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.005212 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.005077 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-ovn-node-metrics-cert\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.009662 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.009638 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj4z9\" (UniqueName: \"kubernetes.io/projected/b19e8ff9-9640-4aee-bab5-ab0ab8c374da-kube-api-access-cj4z9\") pod \"tuned-74htd\" (UID: \"b19e8ff9-9640-4aee-bab5-ab0ab8c374da\") " pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.010739 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.010715 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjtdk\" (UniqueName: \"kubernetes.io/projected/674e54c4-6cb6-4fbc-ba4e-743e240039a7-kube-api-access-wjtdk\") pod \"multus-additional-cni-plugins-5kl2h\" (UID: \"674e54c4-6cb6-4fbc-ba4e-743e240039a7\") " pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.010924 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.010909 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lqg8\" (UniqueName: \"kubernetes.io/projected/5ae6d83b-0067-4e51-b9a8-26b4c2917b78-kube-api-access-6lqg8\") pod \"ovnkube-node-7r54t\" (UID: \"5ae6d83b-0067-4e51-b9a8-26b4c2917b78\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.098156 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.098086 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rnzn8" Jul 07 11:27:08.105089 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.105068 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hvrdw" Jul 07 11:27:08.113797 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.113772 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vbmng" Jul 07 11:27:08.120243 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.120226 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nqrsk" Jul 07 11:27:08.127918 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.127900 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" Jul 07 11:27:08.135635 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.135609 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5kl2h" Jul 07 11:27:08.143780 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.143761 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v9wwc" Jul 07 11:27:08.153264 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.153245 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-74htd" Jul 07 11:27:08.158883 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.158853 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:08.404888 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.404803 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:08.405034 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:08.404947 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:08.405034 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:08.405006 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs podName:e825e10d-32d1-41d4-aa12-1d1f1dbbb429 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:09.404990178 +0000 UTC m=+4.009327003 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs") pod "network-metrics-daemon-9vm97" (UID: "e825e10d-32d1-41d4-aa12-1d1f1dbbb429") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:08.505476 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.505443 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt72t\" (UniqueName: \"kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t\") pod \"network-check-target-zn5pv\" (UID: \"8341686d-bc8f-4896-96cf-558ba701f0e0\") " pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:08.505657 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.505506 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:08.505657 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:08.505610 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:27:08.505657 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:08.505630 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:27:08.505657 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:08.505636 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:08.505657 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:08.505643 2578 projected.go:196] Error preparing data for projected volume kube-api-access-bt72t for pod openshift-network-diagnostics/network-check-target-zn5pv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:08.505874 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:08.505700 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret podName:0eafca62-9210-42e4-8af3-15faf4201332 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:09.505683896 +0000 UTC m=+4.110020714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret") pod "global-pull-secret-syncer-6kjkj" (UID: "0eafca62-9210-42e4-8af3-15faf4201332") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:08.505874 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:08.505721 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t podName:8341686d-bc8f-4896-96cf-558ba701f0e0 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:09.505711691 +0000 UTC m=+4.110048513 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bt72t" (UniqueName: "kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t") pod "network-check-target-zn5pv" (UID: "8341686d-bc8f-4896-96cf-558ba701f0e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:08.829513 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.829440 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:22:07 +0000 UTC" deadline="2028-04-14 00:46:48.894462286 +0000 UTC" Jul 07 11:27:08.829513 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.829510 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15517h19m40.064955754s" Jul 07 11:27:08.891573 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:08.891035 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb19e8ff9_9640_4aee_bab5_ab0ab8c374da.slice/crio-48d4d54345d71785e828e503245f3e615af55627168ed461529090ce95243586 WatchSource:0}: Error finding container 48d4d54345d71785e828e503245f3e615af55627168ed461529090ce95243586: Status 404 returned error can't find the container with id 48d4d54345d71785e828e503245f3e615af55627168ed461529090ce95243586 Jul 07 11:27:08.891991 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:08.891942 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38329b0_4add_42cf_ba11_ff2643797fa1.slice/crio-840c9a5d0890c66e51178193b154c6db9d0d8d5c63867db3dda3732602ba0d82 WatchSource:0}: Error finding container 840c9a5d0890c66e51178193b154c6db9d0d8d5c63867db3dda3732602ba0d82: Status 404 returned error can't find the container with id 840c9a5d0890c66e51178193b154c6db9d0d8d5c63867db3dda3732602ba0d82 Jul 07 11:27:08.895347 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:08.895316 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba1d1827_3e47_4686_8173_7c4619849f84.slice/crio-26039a35117b4e79e37ae11c2972bb30c55c8e9b318a6d8de2744f2ab5734caa WatchSource:0}: Error finding container 26039a35117b4e79e37ae11c2972bb30c55c8e9b318a6d8de2744f2ab5734caa: Status 404 returned error can't find the container with id 26039a35117b4e79e37ae11c2972bb30c55c8e9b318a6d8de2744f2ab5734caa Jul 07 11:27:08.896297 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:08.896126 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d6ea786_e03d_4f74_a080_914765d20aef.slice/crio-c4854795a1619a4df6e08079f6ceb4958f374f5ca6fa78c54d236b9745d35034 WatchSource:0}: Error finding container c4854795a1619a4df6e08079f6ceb4958f374f5ca6fa78c54d236b9745d35034: Status 404 returned error can't find the container with id c4854795a1619a4df6e08079f6ceb4958f374f5ca6fa78c54d236b9745d35034 Jul 07 11:27:08.897238 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:08.897221 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf94c027e_3db7_4431_9687_dcb104f46ac6.slice/crio-cfbcf1a17023aa7815d088d24beecf9d0bd2e4c749a654cfa50a45e99273c902 WatchSource:0}: Error finding container cfbcf1a17023aa7815d088d24beecf9d0bd2e4c749a654cfa50a45e99273c902: Status 404 returned error can't find the container with id cfbcf1a17023aa7815d088d24beecf9d0bd2e4c749a654cfa50a45e99273c902 Jul 07 11:27:08.897859 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:08.897657 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d9acbc8_5af1_4514_bc47_b8c50a06d5a3.slice/crio-b0702ecd949bce35546f3976bc5d65bf0a852d298a55f077490522dd62299c27 WatchSource:0}: Error finding container b0702ecd949bce35546f3976bc5d65bf0a852d298a55f077490522dd62299c27: Status 404 returned error can't find the container with id b0702ecd949bce35546f3976bc5d65bf0a852d298a55f077490522dd62299c27 Jul 07 11:27:08.940363 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.940215 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5kl2h" event={"ID":"674e54c4-6cb6-4fbc-ba4e-743e240039a7","Type":"ContainerStarted","Data":"e712264c83c337ee7b22f2f9e0e5d02c09f874516c02696dbcbc3de795e6999c"} Jul 07 11:27:08.941182 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.941160 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" event={"ID":"f94c027e-3db7-4431-9687-dcb104f46ac6","Type":"ContainerStarted","Data":"cfbcf1a17023aa7815d088d24beecf9d0bd2e4c749a654cfa50a45e99273c902"} Jul 07 11:27:08.942343 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.942311 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vbmng" event={"ID":"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3","Type":"ContainerStarted","Data":"b0702ecd949bce35546f3976bc5d65bf0a852d298a55f077490522dd62299c27"} Jul 07 11:27:08.943468 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.943447 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hvrdw" event={"ID":"d38329b0-4add-42cf-ba11-ff2643797fa1","Type":"ContainerStarted","Data":"840c9a5d0890c66e51178193b154c6db9d0d8d5c63867db3dda3732602ba0d82"} Jul 07 11:27:08.944472 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.944451 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-74htd" event={"ID":"b19e8ff9-9640-4aee-bab5-ab0ab8c374da","Type":"ContainerStarted","Data":"48d4d54345d71785e828e503245f3e615af55627168ed461529090ce95243586"} Jul 07 11:27:08.945499 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.945476 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" event={"ID":"5ae6d83b-0067-4e51-b9a8-26b4c2917b78","Type":"ContainerStarted","Data":"1abe208bb3d50530b8d7fb44bc03346c0af8f924a084655bc283ff1c129df7e6"} Jul 07 11:27:08.947721 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.947693 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rnzn8" event={"ID":"a8319a2e-d262-43be-b188-7d50ac4598be","Type":"ContainerStarted","Data":"38c982b09b336e2b771774404e4bc5a7ac7f7c5d5ff541c3709407f3b1db3bf3"} Jul 07 11:27:08.951375 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.951350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v9wwc" event={"ID":"9d6ea786-e03d-4f74-a080-914765d20aef","Type":"ContainerStarted","Data":"c4854795a1619a4df6e08079f6ceb4958f374f5ca6fa78c54d236b9745d35034"} Jul 07 11:27:08.952938 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:08.952919 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nqrsk" event={"ID":"ba1d1827-3e47-4686-8173-7c4619849f84","Type":"ContainerStarted","Data":"26039a35117b4e79e37ae11c2972bb30c55c8e9b318a6d8de2744f2ab5734caa"} Jul 07 11:27:09.414845 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:09.414725 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:09.415021 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:09.414847 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:09.415021 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:09.414918 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs podName:e825e10d-32d1-41d4-aa12-1d1f1dbbb429 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:11.414896965 +0000 UTC m=+6.019233798 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs") pod "network-metrics-daemon-9vm97" (UID: "e825e10d-32d1-41d4-aa12-1d1f1dbbb429") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:09.515955 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:09.515884 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt72t\" (UniqueName: \"kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t\") pod \"network-check-target-zn5pv\" (UID: \"8341686d-bc8f-4896-96cf-558ba701f0e0\") " pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:09.516138 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:09.516002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:09.516138 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:09.516133 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:09.516248 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:09.516198 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret podName:0eafca62-9210-42e4-8af3-15faf4201332 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:11.516180227 +0000 UTC m=+6.120517049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret") pod "global-pull-secret-syncer-6kjkj" (UID: "0eafca62-9210-42e4-8af3-15faf4201332") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:09.516628 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:09.516610 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:27:09.516695 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:09.516634 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:27:09.516695 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:09.516649 2578 projected.go:196] Error preparing data for projected volume kube-api-access-bt72t for pod openshift-network-diagnostics/network-check-target-zn5pv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:09.516695 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:09.516694 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t podName:8341686d-bc8f-4896-96cf-558ba701f0e0 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:11.516678283 +0000 UTC m=+6.121015108 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bt72t" (UniqueName: "kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t") pod "network-check-target-zn5pv" (UID: "8341686d-bc8f-4896-96cf-558ba701f0e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:09.829999 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:09.829929 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:22:07 +0000 UTC" deadline="2028-02-25 20:18:50.501284557 +0000 UTC" Jul 07 11:27:09.829999 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:09.829970 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14360h51m40.67132083s" Jul 07 11:27:09.933706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:09.933667 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:09.933938 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:09.933710 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:09.933938 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:09.933811 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:09.934094 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:09.933949 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:09.934094 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:09.934069 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:09.934205 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:09.934152 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:09.967033 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:09.966993 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" event={"ID":"85d1b63fc295b631cf34855fb24fcee0","Type":"ContainerStarted","Data":"e03f84c1fa0ac0b7d14c4e0f27e60005649bf63dcbc0fa0f9a651aa0e1daeec1"} Jul 07 11:27:09.978394 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:09.978361 2578 generic.go:356] "Generic (PLEG): container finished" podID="a7d7affa3453ec77ef6377c3782056c0" containerID="307eeffbf5b35406307b4a0bf1c813798cd87e814d6ef64f617c300660b58476" exitCode=0 Jul 07 11:27:09.978588 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:09.978417 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" event={"ID":"a7d7affa3453ec77ef6377c3782056c0","Type":"ContainerDied","Data":"307eeffbf5b35406307b4a0bf1c813798cd87e814d6ef64f617c300660b58476"} Jul 07 11:27:09.982762 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:09.982707 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" podStartSLOduration=2.982686616 podStartE2EDuration="2.982686616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:27:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:09.981670418 +0000 UTC m=+4.586007260" watchObservedRunningTime="2026-07-07 11:27:09.982686616 +0000 UTC m=+4.587023458" Jul 07 11:27:11.003907 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:11.003852 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" event={"ID":"a7d7affa3453ec77ef6377c3782056c0","Type":"ContainerStarted","Data":"609e8c47bc70e8c0bb61bb417988c455d3de80379fded3729f8207766d100308"} Jul 07 11:27:11.019937 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:11.019879 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" podStartSLOduration=4.019859007 podStartE2EDuration="4.019859007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:27:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:11.019611959 +0000 UTC m=+5.623948802" watchObservedRunningTime="2026-07-07 11:27:11.019859007 +0000 UTC m=+5.624195850" Jul 07 11:27:11.433603 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:11.433563 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:11.433783 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:11.433744 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:11.433839 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:11.433807 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs podName:e825e10d-32d1-41d4-aa12-1d1f1dbbb429 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:15.433789326 +0000 UTC m=+10.038126149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs") pod "network-metrics-daemon-9vm97" (UID: "e825e10d-32d1-41d4-aa12-1d1f1dbbb429") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:11.534948 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:11.534048 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:11.534948 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:11.534133 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt72t\" (UniqueName: \"kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t\") pod \"network-check-target-zn5pv\" (UID: \"8341686d-bc8f-4896-96cf-558ba701f0e0\") " pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:11.534948 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:11.534359 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:27:11.534948 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:11.534380 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:27:11.534948 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:11.534393 2578 projected.go:196] Error preparing data for projected volume kube-api-access-bt72t for pod openshift-network-diagnostics/network-check-target-zn5pv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:11.534948 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:11.534452 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t podName:8341686d-bc8f-4896-96cf-558ba701f0e0 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:15.534434241 +0000 UTC m=+10.138771080 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bt72t" (UniqueName: "kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t") pod "network-check-target-zn5pv" (UID: "8341686d-bc8f-4896-96cf-558ba701f0e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:11.534948 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:11.534861 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:11.534948 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:11.534913 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret podName:0eafca62-9210-42e4-8af3-15faf4201332 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:15.534897107 +0000 UTC m=+10.139233930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret") pod "global-pull-secret-syncer-6kjkj" (UID: "0eafca62-9210-42e4-8af3-15faf4201332") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:11.934476 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:11.934438 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:11.934680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:11.934490 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:11.934680 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:11.934602 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:11.934680 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:11.934639 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:11.934846 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:11.934696 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:11.934846 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:11.934776 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:13.935419 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:13.935379 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:13.935938 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:13.935511 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:13.935938 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:13.935900 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:13.936069 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:13.936008 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:13.936069 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:13.936059 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:13.936166 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:13.936126 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:15.467674 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:15.467006 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:15.467674 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:15.467234 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:15.467674 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:15.467307 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs podName:e825e10d-32d1-41d4-aa12-1d1f1dbbb429 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:23.467286424 +0000 UTC m=+18.071623266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs") pod "network-metrics-daemon-9vm97" (UID: "e825e10d-32d1-41d4-aa12-1d1f1dbbb429") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:15.567665 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:15.567624 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt72t\" (UniqueName: \"kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t\") pod \"network-check-target-zn5pv\" (UID: \"8341686d-bc8f-4896-96cf-558ba701f0e0\") " pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:15.567845 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:15.567713 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:15.567845 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:15.567840 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:15.567952 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:15.567903 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret podName:0eafca62-9210-42e4-8af3-15faf4201332 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:23.567883888 +0000 UTC m=+18.172220711 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret") pod "global-pull-secret-syncer-6kjkj" (UID: "0eafca62-9210-42e4-8af3-15faf4201332") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:15.568016 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:15.567993 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:27:15.568016 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:15.568006 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:27:15.568108 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:15.568019 2578 projected.go:196] Error preparing data for projected volume kube-api-access-bt72t for pod openshift-network-diagnostics/network-check-target-zn5pv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:15.568108 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:15.568052 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t podName:8341686d-bc8f-4896-96cf-558ba701f0e0 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:23.56804114 +0000 UTC m=+18.172377967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bt72t" (UniqueName: "kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t") pod "network-check-target-zn5pv" (UID: "8341686d-bc8f-4896-96cf-558ba701f0e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:15.932987 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:15.932953 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:15.933160 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:15.933069 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:15.933216 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:15.933173 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:15.933216 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:15.933086 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:15.933289 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:15.933237 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:15.933349 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:15.933299 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:17.932766 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:17.932727 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:17.933236 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:17.932737 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:17.933236 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:17.932870 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:17.933236 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:17.932935 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:17.933236 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:17.933049 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:17.933236 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:17.933165 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:19.932853 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:19.932774 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:19.932853 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:19.932807 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:19.933309 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:19.932778 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:19.933309 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:19.932900 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:19.933309 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:19.932968 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:19.933309 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:19.933041 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:21.932486 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:21.932449 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:21.932949 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:21.932449 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:21.932949 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:21.932605 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:21.932949 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:21.932672 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:21.932949 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:21.932449 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:21.932949 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:21.932776 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:23.525156 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:23.525119 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:23.525674 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:23.525295 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:23.525674 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:23.525378 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs podName:e825e10d-32d1-41d4-aa12-1d1f1dbbb429 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:39.525357878 +0000 UTC m=+34.129694696 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs") pod "network-metrics-daemon-9vm97" (UID: "e825e10d-32d1-41d4-aa12-1d1f1dbbb429") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:23.626267 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:23.626229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:23.626433 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:23.626305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt72t\" (UniqueName: \"kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t\") pod \"network-check-target-zn5pv\" (UID: \"8341686d-bc8f-4896-96cf-558ba701f0e0\") " pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:23.626433 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:23.626406 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:23.626592 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:23.626490 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret podName:0eafca62-9210-42e4-8af3-15faf4201332 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:39.626467511 +0000 UTC m=+34.230804335 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret") pod "global-pull-secret-syncer-6kjkj" (UID: "0eafca62-9210-42e4-8af3-15faf4201332") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:23.626592 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:23.626411 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:27:23.626592 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:23.626516 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:27:23.626592 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:23.626529 2578 projected.go:196] Error preparing data for projected volume kube-api-access-bt72t for pod openshift-network-diagnostics/network-check-target-zn5pv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:23.626787 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:23.626612 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t podName:8341686d-bc8f-4896-96cf-558ba701f0e0 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:39.626596265 +0000 UTC m=+34.230933084 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bt72t" (UniqueName: "kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t") pod "network-check-target-zn5pv" (UID: "8341686d-bc8f-4896-96cf-558ba701f0e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:23.932481 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:23.932441 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:23.932481 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:23.932477 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:23.932743 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:23.932444 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:23.932743 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:23.932594 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:23.932836 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:23.932744 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:23.932884 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:23.932840 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:25.933115 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:25.933064 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:25.933571 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:25.933157 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:25.933571 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:25.933155 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:25.933571 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:25.933293 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:25.933571 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:25.933329 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:25.933571 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:25.933394 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:27.932467 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:27.932242 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:27.933135 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:27.932264 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:27.933135 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:27.932242 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:27.933135 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:27.932610 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:27.933135 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:27.932727 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:27.933135 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:27.932796 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:28.038685 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:28.038590 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vbmng" event={"ID":"6d9acbc8-5af1-4514-bc47-b8c50a06d5a3","Type":"ContainerStarted","Data":"067646612a63d64a9003aa4787c78903bbcf0f4b9ddfb4158db993d84ace032d"} Jul 07 11:27:28.040026 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:28.039999 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-74htd" event={"ID":"b19e8ff9-9640-4aee-bab5-ab0ab8c374da","Type":"ContainerStarted","Data":"307494bfdd991ddb7f4efe32db4963679a26607e3dc26954be73a17b5eb30b06"} Jul 07 11:27:28.041488 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:28.041460 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rnzn8" event={"ID":"a8319a2e-d262-43be-b188-7d50ac4598be","Type":"ContainerStarted","Data":"568bf9dbf01178e7cb0a1425e625e9f9f126f07218f1371a3d404194ddddec59"} Jul 07 11:27:28.042994 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:28.042972 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v9wwc" event={"ID":"9d6ea786-e03d-4f74-a080-914765d20aef","Type":"ContainerStarted","Data":"0df275c1d033355e14f14b2ddbd3825f160031897e24d317426aedf25675746e"} Jul 07 11:27:28.044313 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:28.044290 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nqrsk" event={"ID":"ba1d1827-3e47-4686-8173-7c4619849f84","Type":"ContainerStarted","Data":"8a0920258600b3c0fd262980a4361889dd126c13abe69905f3240e8d32ae570f"} Jul 07 11:27:28.045749 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:28.045726 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5kl2h" event={"ID":"674e54c4-6cb6-4fbc-ba4e-743e240039a7","Type":"ContainerStarted","Data":"de27b999f824840f25c492d68ef4ebf2f14932d7384dd1708874dd36f26d3c53"} Jul 07 11:27:28.047142 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:28.047124 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" event={"ID":"f94c027e-3db7-4431-9687-dcb104f46ac6","Type":"ContainerStarted","Data":"5291075ddd7a11a1f382eb4028aacd598cdbc1c7490d4cfad6bbb64b43640254"} Jul 07 11:27:28.057947 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:28.057893 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vbmng" podStartSLOduration=4.318878015 podStartE2EDuration="23.057875072s" totalImagesPullingTime="18.738997057s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:27:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:28.057321661 +0000 UTC m=+22.661658560" watchObservedRunningTime="2026-07-07 11:27:28.057875072 +0000 UTC m=+22.662211914" Jul 07 11:27:28.071904 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:28.071844 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nqrsk" podStartSLOduration=3.360817407 podStartE2EDuration="22.071822106s" totalImagesPullingTime="18.711004699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:28.071385117 +0000 UTC m=+22.675721959" watchObservedRunningTime="2026-07-07 11:27:28.071822106 +0000 UTC m=+22.676158948" Jul 07 11:27:28.087568 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:28.087494 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-74htd" podStartSLOduration=3.370026897 podStartE2EDuration="22.087477428s" totalImagesPullingTime="18.717450531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:28.087019973 +0000 UTC m=+22.691356833" watchObservedRunningTime="2026-07-07 11:27:28.087477428 +0000 UTC m=+22.691814268" Jul 07 11:27:28.100847 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:28.100802 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rnzn8" podStartSLOduration=4.393833244 podStartE2EDuration="23.100786889s" totalImagesPullingTime="18.706953645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:27:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:28.100281622 +0000 UTC m=+22.704618462" watchObservedRunningTime="2026-07-07 11:27:28.100786889 +0000 UTC m=+22.705123729" Jul 07 11:27:28.139121 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:28.139057 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v9wwc" podStartSLOduration=7.911270211 podStartE2EDuration="22.139034346s" totalImagesPullingTime="14.227764135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:28.138566407 +0000 UTC m=+22.742903264" watchObservedRunningTime="2026-07-07 11:27:28.139034346 +0000 UTC m=+22.743371188" Jul 07 11:27:29.051190 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.050954 2578 generic.go:356] "Generic (PLEG): container finished" podID="674e54c4-6cb6-4fbc-ba4e-743e240039a7" containerID="de27b999f824840f25c492d68ef4ebf2f14932d7384dd1708874dd36f26d3c53" exitCode=0 Jul 07 11:27:29.051834 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.051034 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5kl2h" event={"ID":"674e54c4-6cb6-4fbc-ba4e-743e240039a7","Type":"ContainerDied","Data":"de27b999f824840f25c492d68ef4ebf2f14932d7384dd1708874dd36f26d3c53"} Jul 07 11:27:29.052759 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.052727 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hvrdw" event={"ID":"d38329b0-4add-42cf-ba11-ff2643797fa1","Type":"ContainerStarted","Data":"11ccd5695de83333b04ac8d70a28b4fb90cc9d031ee4e9fbc94fb48e6bfbbc3b"} Jul 07 11:27:29.055454 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.055435 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:27:29.055865 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.055846 2578 generic.go:356] "Generic (PLEG): container finished" podID="5ae6d83b-0067-4e51-b9a8-26b4c2917b78" containerID="4d6d86da46e4b8a54e7556a7348d3301cd192e3c9a9afb8cd4320069fd92e99f" exitCode=1 Jul 07 11:27:29.055954 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.055926 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" event={"ID":"5ae6d83b-0067-4e51-b9a8-26b4c2917b78","Type":"ContainerStarted","Data":"12c59fbb35ec3dcd5d78ffb574ed58d749c2cd77891b1e9de4d6e2e26c16b7fa"} Jul 07 11:27:29.056019 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.056003 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" event={"ID":"5ae6d83b-0067-4e51-b9a8-26b4c2917b78","Type":"ContainerStarted","Data":"c1169a94398556fa0f39149d80ea4d166289ffc3d5de26b5f34a3092846b5e2b"} Jul 07 11:27:29.056073 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.056060 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" event={"ID":"5ae6d83b-0067-4e51-b9a8-26b4c2917b78","Type":"ContainerStarted","Data":"553279fb8831f3eafab2b0cfd57e46eadc6e91ee8757272c3ab8f7a676b2bbd7"} Jul 07 11:27:29.056123 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.056110 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" event={"ID":"5ae6d83b-0067-4e51-b9a8-26b4c2917b78","Type":"ContainerStarted","Data":"9125e90fdbd201251b21f6ced83a45764e24a1150ac1afc2c1ffdb0c9cee2fa3"} Jul 07 11:27:29.056180 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.056164 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" event={"ID":"5ae6d83b-0067-4e51-b9a8-26b4c2917b78","Type":"ContainerDied","Data":"4d6d86da46e4b8a54e7556a7348d3301cd192e3c9a9afb8cd4320069fd92e99f"} Jul 07 11:27:29.056235 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.056223 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" event={"ID":"5ae6d83b-0067-4e51-b9a8-26b4c2917b78","Type":"ContainerStarted","Data":"1edc2e21aa609b98d4a994f82bd746c5cb1b202c06c7b528ac300c2fe47772fe"} Jul 07 11:27:29.087313 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.087265 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hvrdw" podStartSLOduration=5.372506922 podStartE2EDuration="24.087248939s" totalImagesPullingTime="18.714742017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:27:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:29.086965525 +0000 UTC m=+23.691302367" watchObservedRunningTime="2026-07-07 11:27:29.087248939 +0000 UTC m=+23.691585779" Jul 07 11:27:29.254984 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.254957 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 11:27:29.860147 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.860013 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T11:27:29.254975127Z","UUID":"b04cba73-5e45-4568-a4d3-510d4f5cc984","Handler":null,"Name":"","Endpoint":""} Jul 07 11:27:29.861943 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.861920 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 11:27:29.862071 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.861968 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 11:27:29.933132 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.933100 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:29.933348 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.933100 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:29.933348 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:29.933279 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:29.933348 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:29.933111 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:29.933348 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:29.933309 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:29.933573 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:29.933422 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:30.060726 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:30.060675 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" event={"ID":"f94c027e-3db7-4431-9687-dcb104f46ac6","Type":"ContainerStarted","Data":"30ce0f684fde6e97d600a13581c426dcf64a6e7369067a1bb8a09415ba6ce9ad"} Jul 07 11:27:31.065800 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:31.065573 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" event={"ID":"f94c027e-3db7-4431-9687-dcb104f46ac6","Type":"ContainerStarted","Data":"9fc15382aa2c4f18914f1c57ca0eba6c2f79ab4476928e574b8e406a9687d042"} Jul 07 11:27:31.182877 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:31.182837 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v9wwc" Jul 07 11:27:31.932532 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:31.932493 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:31.932730 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:31.932493 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:31.932730 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:31.932659 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:31.932730 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:31.932662 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:31.932870 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:31.932754 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:31.932870 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:31.932836 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:31.983979 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:31.983936 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v9wwc" Jul 07 11:27:31.984554 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:31.984518 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v9wwc" Jul 07 11:27:32.002647 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:32.002603 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nrckc" podStartSLOduration=4.6969979120000005 podStartE2EDuration="26.002587351s" totalImagesPullingTime="21.305589439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:27:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:31.086618227 +0000 UTC m=+25.690955095" watchObservedRunningTime="2026-07-07 11:27:32.002587351 +0000 UTC m=+26.606924193" Jul 07 11:27:32.071935 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:32.071900 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:27:32.072608 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:32.072342 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" event={"ID":"5ae6d83b-0067-4e51-b9a8-26b4c2917b78","Type":"ContainerStarted","Data":"f159e11ff1617a381fdda4a6d91b6b6852244a1aaf3113ea207d243d5605e585"} Jul 07 11:27:32.073116 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:32.073097 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v9wwc" Jul 07 11:27:33.932708 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:33.932523 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:33.933319 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:33.932524 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:33.933319 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:33.932782 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:33.933319 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:33.932877 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:33.933319 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:33.932574 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:33.933319 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:33.932972 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:34.080500 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:34.080473 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:27:34.080925 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:34.080900 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" event={"ID":"5ae6d83b-0067-4e51-b9a8-26b4c2917b78","Type":"ContainerStarted","Data":"ab9a4f450eac471000f1b095f51f06e44f0f7c45644e3c062266da79eeea8189"} Jul 07 11:27:34.081491 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:34.081420 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:34.081491 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:34.081474 2578 scope.go:117] "RemoveContainer" containerID="4d6d86da46e4b8a54e7556a7348d3301cd192e3c9a9afb8cd4320069fd92e99f" Jul 07 11:27:34.086382 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:34.086351 2578 generic.go:356] "Generic (PLEG): container finished" podID="674e54c4-6cb6-4fbc-ba4e-743e240039a7" containerID="ef17b2f4a03156a278b1d20bc0042900612372e70ce47341ee1feb0b02cdf8cd" exitCode=0 Jul 07 11:27:34.086495 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:34.086422 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5kl2h" event={"ID":"674e54c4-6cb6-4fbc-ba4e-743e240039a7","Type":"ContainerDied","Data":"ef17b2f4a03156a278b1d20bc0042900612372e70ce47341ee1feb0b02cdf8cd"} Jul 07 11:27:34.099332 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:34.099303 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:35.092697 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:35.092668 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:27:35.093159 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:35.093052 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" event={"ID":"5ae6d83b-0067-4e51-b9a8-26b4c2917b78","Type":"ContainerStarted","Data":"5b5a8dbbabe1f63fbefdcc4a55b140f05e3933f980386210b4bedbb5e9a6f1d5"} Jul 07 11:27:35.093314 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:35.093288 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:35.093438 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:35.093362 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:35.109661 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:35.109636 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:27:35.123125 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:35.123075 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" podStartSLOduration=9.953877065 podStartE2EDuration="29.123057885s" totalImagesPullingTime="19.16918082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:35.122592143 +0000 UTC m=+29.726928983" watchObservedRunningTime="2026-07-07 11:27:35.123057885 +0000 UTC m=+29.727394726" Jul 07 11:27:35.272744 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:35.272697 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zn5pv"] Jul 07 11:27:35.272926 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:35.272912 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:35.273055 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:35.273026 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:35.275834 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:35.275808 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6kjkj"] Jul 07 11:27:35.276068 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:35.275960 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:35.276148 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:35.276070 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:35.276695 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:35.276666 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9vm97"] Jul 07 11:27:35.276845 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:35.276796 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:35.276939 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:35.276911 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:36.097904 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:36.097728 2578 generic.go:356] "Generic (PLEG): container finished" podID="674e54c4-6cb6-4fbc-ba4e-743e240039a7" containerID="a3071ddf8004d3832b19fc842ff082b2a1542d35853a45253e9ad8d30a343f55" exitCode=0 Jul 07 11:27:36.098661 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:36.097805 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5kl2h" event={"ID":"674e54c4-6cb6-4fbc-ba4e-743e240039a7","Type":"ContainerDied","Data":"a3071ddf8004d3832b19fc842ff082b2a1542d35853a45253e9ad8d30a343f55"} Jul 07 11:27:36.932427 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:36.932397 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:36.932613 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:36.932398 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:36.932613 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:36.932502 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:36.932721 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:36.932398 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:36.932721 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:36.932614 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:36.932721 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:36.932694 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:38.105024 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:38.104989 2578 generic.go:356] "Generic (PLEG): container finished" podID="674e54c4-6cb6-4fbc-ba4e-743e240039a7" containerID="0eed2cfecf13f92a7ab811d8075daf0b5b808dd9b378f00492f56ef8b8323722" exitCode=0 Jul 07 11:27:38.105735 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:38.105050 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5kl2h" event={"ID":"674e54c4-6cb6-4fbc-ba4e-743e240039a7","Type":"ContainerDied","Data":"0eed2cfecf13f92a7ab811d8075daf0b5b808dd9b378f00492f56ef8b8323722"} Jul 07 11:27:38.932789 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:38.932748 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:38.932978 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:38.932794 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:38.932978 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:38.932762 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:38.932978 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:38.932882 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:38.932978 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:38.932969 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:38.933150 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:38.933060 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:39.544158 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:39.544124 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:39.544627 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:39.544237 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:39.544627 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:39.544298 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs podName:e825e10d-32d1-41d4-aa12-1d1f1dbbb429 nodeName:}" failed. No retries permitted until 2026-07-07 11:28:11.544280539 +0000 UTC m=+66.148617380 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs") pod "network-metrics-daemon-9vm97" (UID: "e825e10d-32d1-41d4-aa12-1d1f1dbbb429") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:27:39.645141 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:39.645099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:39.645326 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:39.645188 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt72t\" (UniqueName: \"kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t\") pod \"network-check-target-zn5pv\" (UID: \"8341686d-bc8f-4896-96cf-558ba701f0e0\") " pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:39.645326 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:39.645272 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:39.645442 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:39.645335 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:27:39.645442 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:39.645344 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret podName:0eafca62-9210-42e4-8af3-15faf4201332 nodeName:}" failed. No retries permitted until 2026-07-07 11:28:11.645328811 +0000 UTC m=+66.249665630 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret") pod "global-pull-secret-syncer-6kjkj" (UID: "0eafca62-9210-42e4-8af3-15faf4201332") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:27:39.645442 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:39.645353 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:27:39.645442 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:39.645362 2578 projected.go:196] Error preparing data for projected volume kube-api-access-bt72t for pod openshift-network-diagnostics/network-check-target-zn5pv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:39.645442 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:39.645410 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t podName:8341686d-bc8f-4896-96cf-558ba701f0e0 nodeName:}" failed. No retries permitted until 2026-07-07 11:28:11.645393288 +0000 UTC m=+66.249730107 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bt72t" (UniqueName: "kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t") pod "network-check-target-zn5pv" (UID: "8341686d-bc8f-4896-96cf-558ba701f0e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:27:40.932512 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:40.932480 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:40.932998 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:40.932479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:40.932998 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:40.932623 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zn5pv" podUID="8341686d-bc8f-4896-96cf-558ba701f0e0" Jul 07 11:27:40.932998 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:40.932488 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:40.932998 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:40.932700 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:27:40.932998 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:40.932798 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kjkj" podUID="0eafca62-9210-42e4-8af3-15faf4201332" Jul 07 11:27:41.440414 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.440384 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeReady" Jul 07 11:27:41.440620 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.440554 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 11:27:41.487948 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.487909 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5g7lc"] Jul 07 11:27:41.490933 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.490908 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:41.493644 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.493607 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fvpk7"] Jul 07 11:27:41.493806 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.493609 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xqnk4\"" Jul 07 11:27:41.493981 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.493940 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 11:27:41.494371 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.494349 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 11:27:41.495473 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.495452 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fvpk7" Jul 07 11:27:41.498656 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.498454 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 11:27:41.498656 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.498607 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7kkm5\"" Jul 07 11:27:41.498656 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.498612 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 11:27:41.498854 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.498683 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 11:27:41.502327 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.502151 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5g7lc"] Jul 07 11:27:41.508026 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.507968 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fvpk7"] Jul 07 11:27:41.661301 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.661211 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:41.661301 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.661270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8a7a959-d967-4b84-b98e-3cc699102bf5-config-volume\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:41.661618 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.661341 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b00dac5-b502-4544-a725-27b6c15b0639-cert\") pod \"ingress-canary-fvpk7\" (UID: \"6b00dac5-b502-4544-a725-27b6c15b0639\") " pod="openshift-ingress-canary/ingress-canary-fvpk7" Jul 07 11:27:41.661618 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.661439 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8a7a959-d967-4b84-b98e-3cc699102bf5-tmp-dir\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:41.661618 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.661473 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf68t\" (UniqueName: \"kubernetes.io/projected/b8a7a959-d967-4b84-b98e-3cc699102bf5-kube-api-access-cf68t\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:41.661618 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.661521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g74jn\" (UniqueName: \"kubernetes.io/projected/6b00dac5-b502-4544-a725-27b6c15b0639-kube-api-access-g74jn\") pod \"ingress-canary-fvpk7\" (UID: \"6b00dac5-b502-4544-a725-27b6c15b0639\") " pod="openshift-ingress-canary/ingress-canary-fvpk7" Jul 07 11:27:41.762717 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.762681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8a7a959-d967-4b84-b98e-3cc699102bf5-tmp-dir\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:41.762717 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.762720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf68t\" (UniqueName: \"kubernetes.io/projected/b8a7a959-d967-4b84-b98e-3cc699102bf5-kube-api-access-cf68t\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:41.762965 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.762762 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g74jn\" (UniqueName: \"kubernetes.io/projected/6b00dac5-b502-4544-a725-27b6c15b0639-kube-api-access-g74jn\") pod \"ingress-canary-fvpk7\" (UID: \"6b00dac5-b502-4544-a725-27b6c15b0639\") " pod="openshift-ingress-canary/ingress-canary-fvpk7" Jul 07 11:27:41.762965 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.762793 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:41.762965 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.762831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8a7a959-d967-4b84-b98e-3cc699102bf5-config-volume\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:41.762965 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.762852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b00dac5-b502-4544-a725-27b6c15b0639-cert\") pod \"ingress-canary-fvpk7\" (UID: \"6b00dac5-b502-4544-a725-27b6c15b0639\") " pod="openshift-ingress-canary/ingress-canary-fvpk7" Jul 07 11:27:41.762965 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:41.762936 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:27:41.763198 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:41.762998 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls podName:b8a7a959-d967-4b84-b98e-3cc699102bf5 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:42.262978166 +0000 UTC m=+36.867315005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls") pod "dns-default-5g7lc" (UID: "b8a7a959-d967-4b84-b98e-3cc699102bf5") : secret "dns-default-metrics-tls" not found Jul 07 11:27:41.763198 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.763147 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8a7a959-d967-4b84-b98e-3cc699102bf5-tmp-dir\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:41.763514 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.763485 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8a7a959-d967-4b84-b98e-3cc699102bf5-config-volume\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:41.768001 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.767973 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b00dac5-b502-4544-a725-27b6c15b0639-cert\") pod \"ingress-canary-fvpk7\" (UID: \"6b00dac5-b502-4544-a725-27b6c15b0639\") " pod="openshift-ingress-canary/ingress-canary-fvpk7" Jul 07 11:27:41.772579 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.772503 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf68t\" (UniqueName: \"kubernetes.io/projected/b8a7a959-d967-4b84-b98e-3cc699102bf5-kube-api-access-cf68t\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:41.772716 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.772614 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g74jn\" (UniqueName: \"kubernetes.io/projected/6b00dac5-b502-4544-a725-27b6c15b0639-kube-api-access-g74jn\") pod \"ingress-canary-fvpk7\" (UID: \"6b00dac5-b502-4544-a725-27b6c15b0639\") " pod="openshift-ingress-canary/ingress-canary-fvpk7" Jul 07 11:27:41.812650 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:41.812606 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fvpk7" Jul 07 11:27:42.268009 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:42.267955 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:42.268364 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:42.268129 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:27:42.268364 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:42.268200 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls podName:b8a7a959-d967-4b84-b98e-3cc699102bf5 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:43.268178519 +0000 UTC m=+37.872515381 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls") pod "dns-default-5g7lc" (UID: "b8a7a959-d967-4b84-b98e-3cc699102bf5") : secret "dns-default-metrics-tls" not found Jul 07 11:27:42.292617 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:42.292581 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fvpk7"] Jul 07 11:27:42.295996 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:27:42.295972 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b00dac5_b502_4544_a725_27b6c15b0639.slice/crio-9f174858c7090e845c94c29610a56e93b229050cb7c26537b1a3b3450c6faab2 WatchSource:0}: Error finding container 9f174858c7090e845c94c29610a56e93b229050cb7c26537b1a3b3450c6faab2: Status 404 returned error can't find the container with id 9f174858c7090e845c94c29610a56e93b229050cb7c26537b1a3b3450c6faab2 Jul 07 11:27:42.932954 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:42.932917 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:27:42.933123 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:42.933110 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:27:42.933183 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:42.933131 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:27:42.936709 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:42.936683 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 11:27:42.936839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:42.936719 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nfmsb\"" Jul 07 11:27:42.936839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:42.936739 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xcgtk\"" Jul 07 11:27:42.936839 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:42.936683 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 11:27:42.936980 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:42.936917 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 11:27:42.937036 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:42.937026 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 11:27:43.121599 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:43.121503 2578 generic.go:356] "Generic (PLEG): container finished" podID="674e54c4-6cb6-4fbc-ba4e-743e240039a7" containerID="1c4834554cedf379c6b2f2631f27ba1ae9963eee5806150d127666f42484f6ae" exitCode=0 Jul 07 11:27:43.121599 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:43.121582 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5kl2h" event={"ID":"674e54c4-6cb6-4fbc-ba4e-743e240039a7","Type":"ContainerDied","Data":"1c4834554cedf379c6b2f2631f27ba1ae9963eee5806150d127666f42484f6ae"} Jul 07 11:27:43.122890 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:43.122852 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fvpk7" event={"ID":"6b00dac5-b502-4544-a725-27b6c15b0639","Type":"ContainerStarted","Data":"9f174858c7090e845c94c29610a56e93b229050cb7c26537b1a3b3450c6faab2"} Jul 07 11:27:43.274524 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:43.274307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:43.274524 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:43.274475 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:27:43.275027 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:43.274666 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls podName:b8a7a959-d967-4b84-b98e-3cc699102bf5 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:45.274642266 +0000 UTC m=+39.878979104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls") pod "dns-default-5g7lc" (UID: "b8a7a959-d967-4b84-b98e-3cc699102bf5") : secret "dns-default-metrics-tls" not found Jul 07 11:27:44.129877 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:44.129802 2578 generic.go:356] "Generic (PLEG): container finished" podID="674e54c4-6cb6-4fbc-ba4e-743e240039a7" containerID="959f3d8d58aab60fa349e161e3f5663fbce1bffe29ac9e254c288565eec5cae5" exitCode=0 Jul 07 11:27:44.129877 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:44.129843 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5kl2h" event={"ID":"674e54c4-6cb6-4fbc-ba4e-743e240039a7","Type":"ContainerDied","Data":"959f3d8d58aab60fa349e161e3f5663fbce1bffe29ac9e254c288565eec5cae5"} Jul 07 11:27:45.135156 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:45.135057 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5kl2h" event={"ID":"674e54c4-6cb6-4fbc-ba4e-743e240039a7","Type":"ContainerStarted","Data":"5a6bcee794839d5ff631f1135a94e8b2813337431e89087a2251e718f423cbd7"} Jul 07 11:27:45.136462 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:45.136440 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fvpk7" event={"ID":"6b00dac5-b502-4544-a725-27b6c15b0639","Type":"ContainerStarted","Data":"5be2837c4e772dc7861d32f7db2287fbf2c53e6822641f56eae6484ef09e4b6a"} Jul 07 11:27:45.157448 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:45.157389 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5kl2h" podStartSLOduration=10.184263679 podStartE2EDuration="39.157370411s" totalImagesPullingTime="28.973106732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:27:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:45.155874187 +0000 UTC m=+39.760211049" watchObservedRunningTime="2026-07-07 11:27:45.157370411 +0000 UTC m=+39.761707253" Jul 07 11:27:45.170634 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:45.170595 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fvpk7" podStartSLOduration=1.631733179 podStartE2EDuration="4.170582427s" totalImagesPullingTime="2.538849248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:45.170274511 +0000 UTC m=+39.774611369" watchObservedRunningTime="2026-07-07 11:27:45.170582427 +0000 UTC m=+39.774919268" Jul 07 11:27:45.291201 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:45.291166 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:45.291399 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:45.291329 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:27:45.291457 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:45.291415 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls podName:b8a7a959-d967-4b84-b98e-3cc699102bf5 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:49.291394751 +0000 UTC m=+43.895731588 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls") pod "dns-default-5g7lc" (UID: "b8a7a959-d967-4b84-b98e-3cc699102bf5") : secret "dns-default-metrics-tls" not found Jul 07 11:27:49.318311 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:49.318272 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:49.318735 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:49.318419 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:27:49.318735 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:49.318485 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls podName:b8a7a959-d967-4b84-b98e-3cc699102bf5 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:57.318466925 +0000 UTC m=+51.922803767 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls") pod "dns-default-5g7lc" (UID: "b8a7a959-d967-4b84-b98e-3cc699102bf5") : secret "dns-default-metrics-tls" not found Jul 07 11:27:57.374206 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:27:57.374170 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:27:57.374712 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:57.374309 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:27:57.374712 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:27:57.374380 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls podName:b8a7a959-d967-4b84-b98e-3cc699102bf5 nodeName:}" failed. No retries permitted until 2026-07-07 11:28:13.374362566 +0000 UTC m=+67.978699384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls") pod "dns-default-5g7lc" (UID: "b8a7a959-d967-4b84-b98e-3cc699102bf5") : secret "dns-default-metrics-tls" not found Jul 07 11:28:07.121177 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:07.121145 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r54t" Jul 07 11:28:11.567705 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.567664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:28:11.570521 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.570502 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 11:28:11.578644 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:28:11.578629 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 11:28:11.578705 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:28:11.578695 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs podName:e825e10d-32d1-41d4-aa12-1d1f1dbbb429 nodeName:}" failed. No retries permitted until 2026-07-07 11:29:15.578678765 +0000 UTC m=+130.183015584 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs") pod "network-metrics-daemon-9vm97" (UID: "e825e10d-32d1-41d4-aa12-1d1f1dbbb429") : secret "metrics-daemon-secret" not found Jul 07 11:28:11.668362 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.668326 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt72t\" (UniqueName: \"kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t\") pod \"network-check-target-zn5pv\" (UID: \"8341686d-bc8f-4896-96cf-558ba701f0e0\") " pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:28:11.668475 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.668390 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:28:11.671648 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.671630 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 11:28:11.671713 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.671661 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 11:28:11.680887 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.680865 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 11:28:11.682088 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.682069 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0eafca62-9210-42e4-8af3-15faf4201332-original-pull-secret\") pod \"global-pull-secret-syncer-6kjkj\" (UID: \"0eafca62-9210-42e4-8af3-15faf4201332\") " pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:28:11.692066 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.692033 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt72t\" (UniqueName: \"kubernetes.io/projected/8341686d-bc8f-4896-96cf-558ba701f0e0-kube-api-access-bt72t\") pod \"network-check-target-zn5pv\" (UID: \"8341686d-bc8f-4896-96cf-558ba701f0e0\") " pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:28:11.744440 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.744404 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kjkj" Jul 07 11:28:11.760100 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.760077 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xcgtk\"" Jul 07 11:28:11.768633 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.768607 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:28:11.876972 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.876942 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6kjkj"] Jul 07 11:28:11.880618 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:28:11.880594 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eafca62_9210_42e4_8af3_15faf4201332.slice/crio-9e0648d93157610145bcbd0f8724102fda38cf24a7975686e9fba308a0363537 WatchSource:0}: Error finding container 9e0648d93157610145bcbd0f8724102fda38cf24a7975686e9fba308a0363537: Status 404 returned error can't find the container with id 9e0648d93157610145bcbd0f8724102fda38cf24a7975686e9fba308a0363537 Jul 07 11:28:11.896864 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:11.896830 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zn5pv"] Jul 07 11:28:11.899965 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:28:11.899935 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8341686d_bc8f_4896_96cf_558ba701f0e0.slice/crio-85f91baa615606d8b9f8789d970d2bb2f6c7c12c23b0e03ce9aca70defedabcc WatchSource:0}: Error finding container 85f91baa615606d8b9f8789d970d2bb2f6c7c12c23b0e03ce9aca70defedabcc: Status 404 returned error can't find the container with id 85f91baa615606d8b9f8789d970d2bb2f6c7c12c23b0e03ce9aca70defedabcc Jul 07 11:28:12.201912 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:12.201877 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zn5pv" event={"ID":"8341686d-bc8f-4896-96cf-558ba701f0e0","Type":"ContainerStarted","Data":"85f91baa615606d8b9f8789d970d2bb2f6c7c12c23b0e03ce9aca70defedabcc"} Jul 07 11:28:12.202884 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:12.202860 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6kjkj" event={"ID":"0eafca62-9210-42e4-8af3-15faf4201332","Type":"ContainerStarted","Data":"9e0648d93157610145bcbd0f8724102fda38cf24a7975686e9fba308a0363537"} Jul 07 11:28:13.391378 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:13.391333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:28:13.391872 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:28:13.391462 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:28:13.391872 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:28:13.391560 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls podName:b8a7a959-d967-4b84-b98e-3cc699102bf5 nodeName:}" failed. No retries permitted until 2026-07-07 11:28:45.39152131 +0000 UTC m=+99.995858136 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls") pod "dns-default-5g7lc" (UID: "b8a7a959-d967-4b84-b98e-3cc699102bf5") : secret "dns-default-metrics-tls" not found Jul 07 11:28:17.216800 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:17.216761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zn5pv" event={"ID":"8341686d-bc8f-4896-96cf-558ba701f0e0","Type":"ContainerStarted","Data":"794bbd9ea2b08a3e1acc1a3498dcb02731ed905dd847ce3e4b8f9a7f3d3209c3"} Jul 07 11:28:17.217362 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:17.216840 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:28:17.218271 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:17.218237 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6kjkj" event={"ID":"0eafca62-9210-42e4-8af3-15faf4201332","Type":"ContainerStarted","Data":"5945acfab39d4735e8866408b014f166cebf0407f8ebca2f33c371b31deeeaae"} Jul 07 11:28:17.234613 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:17.234564 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zn5pv" podStartSLOduration=67.659440033 podStartE2EDuration="1m12.234528973s" totalImagesPullingTime="4.57508894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:27:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:28:17.234422164 +0000 UTC m=+71.838759005" watchObservedRunningTime="2026-07-07 11:28:17.234528973 +0000 UTC m=+71.838865815" Jul 07 11:28:17.249175 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:17.249121 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6kjkj" podStartSLOduration=66.65171433 podStartE2EDuration="1m11.24910584s" totalImagesPullingTime="4.59739151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:28:17.248629993 +0000 UTC m=+71.852966857" watchObservedRunningTime="2026-07-07 11:28:17.24910584 +0000 UTC m=+71.853442694" Jul 07 11:28:38.445200 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.445087 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd"] Jul 07 11:28:38.447386 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.447363 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:38.449846 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.449828 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 11:28:38.449960 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.449827 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 11:28:38.449960 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.449828 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 07 11:28:38.449960 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.449865 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 11:28:38.458227 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.458204 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd"] Jul 07 11:28:38.461745 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.461721 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6"] Jul 07 11:28:38.463726 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.463710 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.466028 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.466007 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 07 11:28:38.466150 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.466069 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 07 11:28:38.466150 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.466009 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 07 11:28:38.466267 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.466212 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 07 11:28:38.479440 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.479418 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6"] Jul 07 11:28:38.558725 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.558685 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2-tmp\") pod \"klusterlet-addon-workmgr-844dcd896-sm7kd\" (UID: \"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:38.558725 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.558727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh9zt\" (UniqueName: \"kubernetes.io/projected/65910b63-9b97-49ae-b758-ff9de6cc7a52-kube-api-access-fh9zt\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.558953 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.558748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/65910b63-9b97-49ae-b758-ff9de6cc7a52-additional-service-ca\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.558953 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.558769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2-klusterlet-config\") pod \"klusterlet-addon-workmgr-844dcd896-sm7kd\" (UID: \"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:38.558953 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.558882 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/65910b63-9b97-49ae-b758-ff9de6cc7a52-ca\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.558953 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.558911 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsxzl\" (UniqueName: \"kubernetes.io/projected/ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2-kube-api-access-nsxzl\") pod \"klusterlet-addon-workmgr-844dcd896-sm7kd\" (UID: \"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:38.558953 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.558929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/65910b63-9b97-49ae-b758-ff9de6cc7a52-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.558953 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.558949 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/65910b63-9b97-49ae-b758-ff9de6cc7a52-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.559163 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.559017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/65910b63-9b97-49ae-b758-ff9de6cc7a52-hub\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.660077 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.660041 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/65910b63-9b97-49ae-b758-ff9de6cc7a52-hub\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.660077 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.660083 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2-tmp\") pod \"klusterlet-addon-workmgr-844dcd896-sm7kd\" (UID: \"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:38.660334 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.660105 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh9zt\" (UniqueName: \"kubernetes.io/projected/65910b63-9b97-49ae-b758-ff9de6cc7a52-kube-api-access-fh9zt\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.660334 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.660133 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/65910b63-9b97-49ae-b758-ff9de6cc7a52-additional-service-ca\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.660334 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.660159 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2-klusterlet-config\") pod \"klusterlet-addon-workmgr-844dcd896-sm7kd\" (UID: \"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:38.660334 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.660214 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/65910b63-9b97-49ae-b758-ff9de6cc7a52-ca\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.660334 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.660247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsxzl\" (UniqueName: \"kubernetes.io/projected/ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2-kube-api-access-nsxzl\") pod \"klusterlet-addon-workmgr-844dcd896-sm7kd\" (UID: \"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:38.660334 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.660274 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/65910b63-9b97-49ae-b758-ff9de6cc7a52-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.660334 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.660305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/65910b63-9b97-49ae-b758-ff9de6cc7a52-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.660725 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.660531 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2-tmp\") pod \"klusterlet-addon-workmgr-844dcd896-sm7kd\" (UID: \"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:38.660924 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.660893 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/65910b63-9b97-49ae-b758-ff9de6cc7a52-additional-service-ca\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.662765 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.662736 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2-klusterlet-config\") pod \"klusterlet-addon-workmgr-844dcd896-sm7kd\" (UID: \"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:38.663034 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.663013 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/65910b63-9b97-49ae-b758-ff9de6cc7a52-ca\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.663100 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.663087 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/65910b63-9b97-49ae-b758-ff9de6cc7a52-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.663137 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.663121 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/65910b63-9b97-49ae-b758-ff9de6cc7a52-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.663137 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.663120 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/65910b63-9b97-49ae-b758-ff9de6cc7a52-hub\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.668341 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.668321 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh9zt\" (UniqueName: \"kubernetes.io/projected/65910b63-9b97-49ae-b758-ff9de6cc7a52-kube-api-access-fh9zt\") pod \"cluster-proxy-proxy-agent-564dc7fc85-xgkl6\" (UID: \"65910b63-9b97-49ae-b758-ff9de6cc7a52\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.668666 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.668644 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsxzl\" (UniqueName: \"kubernetes.io/projected/ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2-kube-api-access-nsxzl\") pod \"klusterlet-addon-workmgr-844dcd896-sm7kd\" (UID: \"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:38.756578 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.756467 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:38.786021 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.785988 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:28:38.888657 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.888617 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd"] Jul 07 11:28:38.892991 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:28:38.892960 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec55d7ca_ee8c_4b08_9a4c_55d82d5db1b2.slice/crio-e1e2e51f94fec1f94f887ee612fa4a97db3964da11deeca913287eb7dcbeb5df WatchSource:0}: Error finding container e1e2e51f94fec1f94f887ee612fa4a97db3964da11deeca913287eb7dcbeb5df: Status 404 returned error can't find the container with id e1e2e51f94fec1f94f887ee612fa4a97db3964da11deeca913287eb7dcbeb5df Jul 07 11:28:38.916004 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:38.915975 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6"] Jul 07 11:28:38.920073 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:28:38.920037 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65910b63_9b97_49ae_b758_ff9de6cc7a52.slice/crio-c20897d739b06f7303b77b1eb6a76db0085436bd742f3d7716c7a995b1acd57e WatchSource:0}: Error finding container c20897d739b06f7303b77b1eb6a76db0085436bd742f3d7716c7a995b1acd57e: Status 404 returned error can't find the container with id c20897d739b06f7303b77b1eb6a76db0085436bd742f3d7716c7a995b1acd57e Jul 07 11:28:39.273589 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:39.273552 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" event={"ID":"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2","Type":"ContainerStarted","Data":"e1e2e51f94fec1f94f887ee612fa4a97db3964da11deeca913287eb7dcbeb5df"} Jul 07 11:28:39.274579 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:39.274557 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" event={"ID":"65910b63-9b97-49ae-b758-ff9de6cc7a52","Type":"ContainerStarted","Data":"c20897d739b06f7303b77b1eb6a76db0085436bd742f3d7716c7a995b1acd57e"} Jul 07 11:28:44.292612 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:44.292506 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" event={"ID":"65910b63-9b97-49ae-b758-ff9de6cc7a52","Type":"ContainerStarted","Data":"ef9a9640abe5112772b19d9ec12a0701926026ccb0f02d070e8280a355fe82d6"} Jul 07 11:28:44.292612 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:44.292582 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" event={"ID":"65910b63-9b97-49ae-b758-ff9de6cc7a52","Type":"ContainerStarted","Data":"6d0b15a0fd3983d5627c4eaef8cc7b389bd83649ca1859dae468cbd751969315"} Jul 07 11:28:44.292612 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:44.292604 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" event={"ID":"65910b63-9b97-49ae-b758-ff9de6cc7a52","Type":"ContainerStarted","Data":"0dc9ede90b014f62600d692439ef87be64c6021ead270d621e49a727c49f3a09"} Jul 07 11:28:44.293992 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:44.293968 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" event={"ID":"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2","Type":"ContainerStarted","Data":"b5e8960986d984fab066d59bb2d83639f638737387019a9873052f3e5c02af11"} Jul 07 11:28:44.294172 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:44.294159 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:44.295818 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:44.295797 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:28:44.311366 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:44.311319 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" podStartSLOduration=1.2764110309999999 podStartE2EDuration="6.311303528s" totalImagesPullingTime="5.034892497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:28:44.309945667 +0000 UTC m=+98.914282510" watchObservedRunningTime="2026-07-07 11:28:44.311303528 +0000 UTC m=+98.915640368" Jul 07 11:28:44.326272 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:44.326227 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" podStartSLOduration=1.267467852 podStartE2EDuration="6.326212233s" totalImagesPullingTime="5.058744381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:28:44.325305047 +0000 UTC m=+98.929641887" watchObservedRunningTime="2026-07-07 11:28:44.326212233 +0000 UTC m=+98.930549074" Jul 07 11:28:45.416234 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:45.416196 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:28:45.416773 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:28:45.416605 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:28:45.416773 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:28:45.416658 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls podName:b8a7a959-d967-4b84-b98e-3cc699102bf5 nodeName:}" failed. No retries permitted until 2026-07-07 11:29:49.416641191 +0000 UTC m=+164.020978022 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls") pod "dns-default-5g7lc" (UID: "b8a7a959-d967-4b84-b98e-3cc699102bf5") : secret "dns-default-metrics-tls" not found Jul 07 11:28:48.222886 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:28:48.222858 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zn5pv" Jul 07 11:29:15.636695 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:29:15.636652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:29:15.637164 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:29:15.636803 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 11:29:15.637164 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:29:15.636875 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs podName:e825e10d-32d1-41d4-aa12-1d1f1dbbb429 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:17.636857054 +0000 UTC m=+252.241193874 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs") pod "network-metrics-daemon-9vm97" (UID: "e825e10d-32d1-41d4-aa12-1d1f1dbbb429") : secret "metrics-daemon-secret" not found Jul 07 11:29:44.504783 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:29:44.504743 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5g7lc" podUID="b8a7a959-d967-4b84-b98e-3cc699102bf5" Jul 07 11:29:45.469100 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:29:45.469071 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5g7lc" Jul 07 11:29:45.951628 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:29:45.951595 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:29:49.465759 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:29:49.465720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:29:49.466170 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:29:49.465873 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:29:49.466170 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:29:49.465944 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls podName:b8a7a959-d967-4b84-b98e-3cc699102bf5 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:51.465927969 +0000 UTC m=+286.070264788 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls") pod "dns-default-5g7lc" (UID: "b8a7a959-d967-4b84-b98e-3cc699102bf5") : secret "dns-default-metrics-tls" not found Jul 07 11:29:58.932706 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:29:58.932671 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:30:44.295390 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:30:44.295341 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" podUID="ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused" Jul 07 11:30:44.633336 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:30:44.633302 2578 generic.go:356] "Generic (PLEG): container finished" podID="ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2" containerID="b5e8960986d984fab066d59bb2d83639f638737387019a9873052f3e5c02af11" exitCode=1 Jul 07 11:30:44.633482 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:30:44.633370 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" event={"ID":"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2","Type":"ContainerDied","Data":"b5e8960986d984fab066d59bb2d83639f638737387019a9873052f3e5c02af11"} Jul 07 11:30:44.633707 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:30:44.633693 2578 scope.go:117] "RemoveContainer" containerID="b5e8960986d984fab066d59bb2d83639f638737387019a9873052f3e5c02af11" Jul 07 11:30:45.637736 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:30:45.637697 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" event={"ID":"ec55d7ca-ee8c-4b08-9a4c-55d82d5db1b2","Type":"ContainerStarted","Data":"a8998b31678fd71ab0fe3588a75ff6ee01839ca069ab0635f28d1b4492c065a4"} Jul 07 11:30:45.638211 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:30:45.637975 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:30:45.638658 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:30:45.638639 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844dcd896-sm7kd" Jul 07 11:31:17.666024 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:17.665962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:31:17.666529 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:17.666129 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 11:31:17.666529 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:17.666199 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs podName:e825e10d-32d1-41d4-aa12-1d1f1dbbb429 nodeName:}" failed. No retries permitted until 2026-07-07 11:33:19.666180083 +0000 UTC m=+374.270516903 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs") pod "network-metrics-daemon-9vm97" (UID: "e825e10d-32d1-41d4-aa12-1d1f1dbbb429") : secret "metrics-daemon-secret" not found Jul 07 11:31:30.064950 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.064866 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nbkgr"] Jul 07 11:31:30.067135 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.067115 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.072388 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.072346 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 11:31:30.072388 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.072348 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 11:31:30.072591 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.072400 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 11:31:30.073220 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.073202 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bv7rg\"" Jul 07 11:31:30.073288 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.073222 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 11:31:30.089030 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.089000 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nbkgr"] Jul 07 11:31:30.249683 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.249635 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-crio-socket\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.249683 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.249689 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.249917 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.249748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-data-volume\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.249917 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.249768 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.249917 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.249791 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g6jp\" (UniqueName: \"kubernetes.io/projected/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-api-access-2g6jp\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.350601 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.350501 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.350601 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.350561 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-data-volume\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.350815 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:30.350634 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 11:31:30.350815 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:30.350725 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm podName:9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:30.850702492 +0000 UTC m=+265.455039319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nbkgr" (UID: "9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441") : configmap "kube-rbac-proxy" not found Jul 07 11:31:30.350815 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.350748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.350815 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.350780 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g6jp\" (UniqueName: \"kubernetes.io/projected/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-api-access-2g6jp\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.351028 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.350871 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-crio-socket\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.351028 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:30.350896 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 11:31:30.351028 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.350957 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-crio-socket\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.351028 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:30.350968 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls podName:9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:30.850954614 +0000 UTC m=+265.455291438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nbkgr" (UID: "9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441") : secret "insights-runtime-extractor-tls" not found Jul 07 11:31:30.351293 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.351275 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-data-volume\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.362968 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.362941 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g6jp\" (UniqueName: \"kubernetes.io/projected/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-api-access-2g6jp\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.855271 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.855238 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.855435 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:30.855283 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:30.855435 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:30.855352 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 11:31:30.855435 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:30.855387 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 11:31:30.855435 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:30.855420 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm podName:9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:31.855402639 +0000 UTC m=+266.459739464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nbkgr" (UID: "9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441") : configmap "kube-rbac-proxy" not found Jul 07 11:31:30.855435 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:30.855436 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls podName:9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:31.855425516 +0000 UTC m=+266.459762334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nbkgr" (UID: "9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441") : secret "insights-runtime-extractor-tls" not found Jul 07 11:31:31.861421 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:31.861368 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:31.861832 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:31.861510 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 11:31:31.861832 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:31.861587 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls podName:9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:33.861572916 +0000 UTC m=+268.465909734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nbkgr" (UID: "9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441") : secret "insights-runtime-extractor-tls" not found Jul 07 11:31:31.861832 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:31.861605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:31.861832 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:31.861681 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 11:31:31.861832 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:31.861703 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm podName:9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:33.861695623 +0000 UTC m=+268.466032442 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nbkgr" (UID: "9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441") : configmap "kube-rbac-proxy" not found Jul 07 11:31:32.028826 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:32.028800 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nqrsk_ba1d1827-3e47-4686-8173-7c4619849f84/dns-node-resolver/0.log" Jul 07 11:31:33.228579 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:33.228532 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rnzn8_a8319a2e-d262-43be-b188-7d50ac4598be/node-ca/0.log" Jul 07 11:31:33.842480 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:33.842454 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fvpk7_6b00dac5-b502-4544-a725-27b6c15b0639/serve-healthcheck-canary/0.log" Jul 07 11:31:33.875811 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:33.875781 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:33.875944 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:33.875832 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:33.875944 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:33.875897 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 11:31:33.876021 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:33.875961 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm podName:9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:37.875948293 +0000 UTC m=+272.480285113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nbkgr" (UID: "9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441") : configmap "kube-rbac-proxy" not found Jul 07 11:31:33.876021 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:33.875978 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 11:31:33.876102 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:33.876028 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls podName:9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:37.876015499 +0000 UTC m=+272.480352318 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nbkgr" (UID: "9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441") : secret "insights-runtime-extractor-tls" not found Jul 07 11:31:37.906426 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:37.906388 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:37.906865 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:37.906435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:37.906865 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:37.906503 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 11:31:37.906865 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:37.906522 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 11:31:37.906865 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:37.906602 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm podName:9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:45.906580427 +0000 UTC m=+280.510917246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nbkgr" (UID: "9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441") : configmap "kube-rbac-proxy" not found Jul 07 11:31:37.906865 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:37.906651 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls podName:9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:45.906643225 +0000 UTC m=+280.510980043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nbkgr" (UID: "9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441") : secret "insights-runtime-extractor-tls" not found Jul 07 11:31:45.967995 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:45.967954 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:45.968409 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:45.968031 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:45.968409 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:45.968094 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 11:31:45.968409 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:45.968152 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls podName:9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441 nodeName:}" failed. No retries permitted until 2026-07-07 11:32:01.968135854 +0000 UTC m=+296.572472673 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nbkgr" (UID: "9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441") : secret "insights-runtime-extractor-tls" not found Jul 07 11:31:45.968562 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:45.968461 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:31:48.470578 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:31:48.470495 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5g7lc" podUID="b8a7a959-d967-4b84-b98e-3cc699102bf5" Jul 07 11:31:48.815662 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:48.815570 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5g7lc" Jul 07 11:31:51.509117 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:51.509071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:31:51.511391 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:51.511367 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a7a959-d967-4b84-b98e-3cc699102bf5-metrics-tls\") pod \"dns-default-5g7lc\" (UID: \"b8a7a959-d967-4b84-b98e-3cc699102bf5\") " pod="openshift-dns/dns-default-5g7lc" Jul 07 11:31:51.518873 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:51.518854 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xqnk4\"" Jul 07 11:31:51.527186 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:51.527171 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5g7lc" Jul 07 11:31:51.654274 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:51.654238 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5g7lc"] Jul 07 11:31:51.657975 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:31:51.657950 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a7a959_d967_4b84_b98e_3cc699102bf5.slice/crio-aeb57727b19f537cf9737147496895e8726edc6924ec85a90571255294e04322 WatchSource:0}: Error finding container aeb57727b19f537cf9737147496895e8726edc6924ec85a90571255294e04322: Status 404 returned error can't find the container with id aeb57727b19f537cf9737147496895e8726edc6924ec85a90571255294e04322 Jul 07 11:31:51.825823 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:51.825741 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5g7lc" event={"ID":"b8a7a959-d967-4b84-b98e-3cc699102bf5","Type":"ContainerStarted","Data":"aeb57727b19f537cf9737147496895e8726edc6924ec85a90571255294e04322"} Jul 07 11:31:53.834407 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:53.834375 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5g7lc" event={"ID":"b8a7a959-d967-4b84-b98e-3cc699102bf5","Type":"ContainerStarted","Data":"3260c7ab00195b31c7f3ff46aae3a932b2b2e36b44bb66b3e95d1192a2125514"} Jul 07 11:31:53.834818 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:53.834425 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5g7lc" event={"ID":"b8a7a959-d967-4b84-b98e-3cc699102bf5","Type":"ContainerStarted","Data":"ee8c8d956bb4400b5cc9bbcbbdc1976193ae01bbf58bc0a0ec546c836816847a"} Jul 07 11:31:53.834818 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:53.834494 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5g7lc" Jul 07 11:31:53.864350 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:31:53.864303 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5g7lc" podStartSLOduration=251.26198073 podStartE2EDuration="4m12.864286987s" totalImagesPullingTime="1.602306257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:31:53.861183081 +0000 UTC m=+288.465519923" watchObservedRunningTime="2026-07-07 11:31:53.864286987 +0000 UTC m=+288.468623805" Jul 07 11:32:01.933159 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:32:01.933111 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9vm97" podUID="e825e10d-32d1-41d4-aa12-1d1f1dbbb429" Jul 07 11:32:01.980785 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:01.980744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:32:01.983077 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:01.983057 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nbkgr\" (UID: \"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441\") " pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:32:02.175990 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:02.175954 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nbkgr" Jul 07 11:32:02.302744 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:02.302710 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nbkgr"] Jul 07 11:32:02.305933 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:32:02.305911 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fc4fbc8_0d3c_497a_a4e8_5d1b5f623441.slice/crio-d4b672bb075f1f44e7a9749374bde2fda6665fb080ce70d3d2e15ab551b1cc04 WatchSource:0}: Error finding container d4b672bb075f1f44e7a9749374bde2fda6665fb080ce70d3d2e15ab551b1cc04: Status 404 returned error can't find the container with id d4b672bb075f1f44e7a9749374bde2fda6665fb080ce70d3d2e15ab551b1cc04 Jul 07 11:32:02.864454 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:02.864370 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nbkgr" event={"ID":"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441","Type":"ContainerStarted","Data":"d4b672bb075f1f44e7a9749374bde2fda6665fb080ce70d3d2e15ab551b1cc04"} Jul 07 11:32:03.840128 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:03.840096 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5g7lc" Jul 07 11:32:05.970952 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:05.970789 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:32:05.985142 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:05.971039 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:32:05.985142 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:05.977217 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 11:32:06.841186 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.841156 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f7c65cbb6-28xbm"] Jul 07 11:32:06.844185 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.844167 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:06.846818 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.846792 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 11:32:06.847072 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.847053 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 11:32:06.847794 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.847777 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mjmt6\"" Jul 07 11:32:06.847884 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.847846 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 11:32:06.848047 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.848026 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 11:32:06.848166 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.848083 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 11:32:06.848166 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.848134 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 11:32:06.848391 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.848376 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 11:32:06.853156 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.853131 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 11:32:06.856306 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.856283 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f7c65cbb6-28xbm"] Jul 07 11:32:06.878162 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.878131 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nbkgr" event={"ID":"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441","Type":"ContainerStarted","Data":"d034c749856f795264b557661d1c2640380ba218eeb2704a4b3e166e074d3e57"} Jul 07 11:32:06.878312 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.878178 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nbkgr" event={"ID":"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441","Type":"ContainerStarted","Data":"7f35ef9528cfdd4ab9070e5b76bb458485f506aba2f5cd0f1a020f07916553b5"} Jul 07 11:32:06.919481 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.919451 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-console-config\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:06.919645 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.919506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-oauth-serving-cert\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:06.919645 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.919556 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/64854197-cf10-42f6-baea-2e0e5274b921-console-oauth-config\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:06.919645 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.919626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/64854197-cf10-42f6-baea-2e0e5274b921-console-serving-cert\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:06.919800 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.919663 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s87mw\" (UniqueName: \"kubernetes.io/projected/64854197-cf10-42f6-baea-2e0e5274b921-kube-api-access-s87mw\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:06.919800 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.919689 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-trusted-ca-bundle\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:06.919800 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:06.919731 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-service-ca\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.020309 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.020273 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-oauth-serving-cert\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.020782 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.020332 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/64854197-cf10-42f6-baea-2e0e5274b921-console-oauth-config\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.020782 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.020367 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/64854197-cf10-42f6-baea-2e0e5274b921-console-serving-cert\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.020782 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.020386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s87mw\" (UniqueName: \"kubernetes.io/projected/64854197-cf10-42f6-baea-2e0e5274b921-kube-api-access-s87mw\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.020782 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.020401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-trusted-ca-bundle\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.020782 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.020437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-service-ca\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.020782 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.020507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-console-config\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.021104 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.020967 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-oauth-serving-cert\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.021166 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.021143 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-service-ca\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.021286 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.021266 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-trusted-ca-bundle\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.021344 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.021273 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-console-config\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.023251 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.023224 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/64854197-cf10-42f6-baea-2e0e5274b921-console-oauth-config\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.023352 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.023310 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/64854197-cf10-42f6-baea-2e0e5274b921-console-serving-cert\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.029403 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.029382 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s87mw\" (UniqueName: \"kubernetes.io/projected/64854197-cf10-42f6-baea-2e0e5274b921-kube-api-access-s87mw\") pod \"console-7f7c65cbb6-28xbm\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.153704 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.153661 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:07.291095 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.291066 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f7c65cbb6-28xbm"] Jul 07 11:32:07.296089 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:32:07.296053 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64854197_cf10_42f6_baea_2e0e5274b921.slice/crio-3ba66d3a54a0a71c35002f08cccde4ac67fb3e14d52841f31a2fc8dd61deeaff WatchSource:0}: Error finding container 3ba66d3a54a0a71c35002f08cccde4ac67fb3e14d52841f31a2fc8dd61deeaff: Status 404 returned error can't find the container with id 3ba66d3a54a0a71c35002f08cccde4ac67fb3e14d52841f31a2fc8dd61deeaff Jul 07 11:32:07.298480 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.298460 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:32:07.882929 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:07.882892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7c65cbb6-28xbm" event={"ID":"64854197-cf10-42f6-baea-2e0e5274b921","Type":"ContainerStarted","Data":"3ba66d3a54a0a71c35002f08cccde4ac67fb3e14d52841f31a2fc8dd61deeaff"} Jul 07 11:32:10.654506 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.654477 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gq77g"] Jul 07 11:32:10.659109 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.659090 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.661834 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.661817 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 11:32:10.662504 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.662481 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 11:32:10.662762 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.662748 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 11:32:10.665882 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.665866 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 11:32:10.666178 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.666162 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 11:32:10.666178 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.666173 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9pw78\"" Jul 07 11:32:10.671494 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.671474 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 11:32:10.753751 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.753719 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-accelerators-collector-config\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.753900 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.753779 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-textfile\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.753900 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.753803 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-wtmp\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.753900 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.753818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87fd6ff0-9ee3-4be5-8f10-668b5e330994-metrics-client-ca\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.753900 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.753836 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/87fd6ff0-9ee3-4be5-8f10-668b5e330994-root\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.753900 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.753870 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87fd6ff0-9ee3-4be5-8f10-668b5e330994-sys\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.754098 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.753929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.754098 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.753960 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gvsv\" (UniqueName: \"kubernetes.io/projected/87fd6ff0-9ee3-4be5-8f10-668b5e330994-kube-api-access-9gvsv\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.754098 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.753985 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-tls\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.854835 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.854796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-tls\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.854835 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.854838 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-accelerators-collector-config\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855074 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.854877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-textfile\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855074 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.854893 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-wtmp\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855074 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.854910 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87fd6ff0-9ee3-4be5-8f10-668b5e330994-metrics-client-ca\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855074 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.854925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/87fd6ff0-9ee3-4be5-8f10-668b5e330994-root\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855074 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.854956 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87fd6ff0-9ee3-4be5-8f10-668b5e330994-sys\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855074 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.855002 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87fd6ff0-9ee3-4be5-8f10-668b5e330994-sys\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855074 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.855036 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/87fd6ff0-9ee3-4be5-8f10-668b5e330994-root\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855402 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.855070 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-wtmp\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855402 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.855113 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855402 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.855157 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gvsv\" (UniqueName: \"kubernetes.io/projected/87fd6ff0-9ee3-4be5-8f10-668b5e330994-kube-api-access-9gvsv\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855402 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.855248 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-textfile\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855705 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.855674 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-accelerators-collector-config\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.855956 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.855937 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87fd6ff0-9ee3-4be5-8f10-668b5e330994-metrics-client-ca\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.857224 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.857202 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-tls\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.857312 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.857216 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87fd6ff0-9ee3-4be5-8f10-668b5e330994-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.863342 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.863322 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gvsv\" (UniqueName: \"kubernetes.io/projected/87fd6ff0-9ee3-4be5-8f10-668b5e330994-kube-api-access-9gvsv\") pod \"node-exporter-gq77g\" (UID: \"87fd6ff0-9ee3-4be5-8f10-668b5e330994\") " pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.895452 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.895360 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7c65cbb6-28xbm" event={"ID":"64854197-cf10-42f6-baea-2e0e5274b921","Type":"ContainerStarted","Data":"48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584"} Jul 07 11:32:10.897346 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.897319 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nbkgr" event={"ID":"9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441","Type":"ContainerStarted","Data":"47fe76815580f35ea7da9feecc33e920c15bb689a6edce9f93a04cd9d271e453"} Jul 07 11:32:10.919112 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.919065 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f7c65cbb6-28xbm" podStartSLOduration=1.598310436 podStartE2EDuration="4.919049984s" totalImagesPullingTime="3.320739548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:32:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:32:10.918200262 +0000 UTC m=+305.522537102" watchObservedRunningTime="2026-07-07 11:32:10.919049984 +0000 UTC m=+305.523386864" Jul 07 11:32:10.939758 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.939708 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nbkgr" podStartSLOduration=32.734261900999996 podStartE2EDuration="40.939694122s" totalImagesPullingTime="8.205432221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:31:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:32:10.938839117 +0000 UTC m=+305.543175994" watchObservedRunningTime="2026-07-07 11:32:10.939694122 +0000 UTC m=+305.544030963" Jul 07 11:32:10.974086 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:10.974059 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gq77g" Jul 07 11:32:10.986434 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:32:10.986409 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87fd6ff0_9ee3_4be5_8f10_668b5e330994.slice/crio-878c049102f6ec9276e9b93198c76d3e6ac167fc3b4c9d585e85d14f25d8fc25 WatchSource:0}: Error finding container 878c049102f6ec9276e9b93198c76d3e6ac167fc3b4c9d585e85d14f25d8fc25: Status 404 returned error can't find the container with id 878c049102f6ec9276e9b93198c76d3e6ac167fc3b4c9d585e85d14f25d8fc25 Jul 07 11:32:11.902357 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:11.902321 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gq77g" event={"ID":"87fd6ff0-9ee3-4be5-8f10-668b5e330994","Type":"ContainerStarted","Data":"878c049102f6ec9276e9b93198c76d3e6ac167fc3b4c9d585e85d14f25d8fc25"} Jul 07 11:32:12.910835 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:12.910803 2578 generic.go:356] "Generic (PLEG): container finished" podID="87fd6ff0-9ee3-4be5-8f10-668b5e330994" containerID="4ddfe7bab3ab065c9e5ce82d1f78327e802899123750167ed8b30ea2a9ef080a" exitCode=0 Jul 07 11:32:12.911219 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:12.910850 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gq77g" event={"ID":"87fd6ff0-9ee3-4be5-8f10-668b5e330994","Type":"ContainerDied","Data":"4ddfe7bab3ab065c9e5ce82d1f78327e802899123750167ed8b30ea2a9ef080a"} Jul 07 11:32:13.918973 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:13.918937 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gq77g" event={"ID":"87fd6ff0-9ee3-4be5-8f10-668b5e330994","Type":"ContainerStarted","Data":"6b536ece9b7a01c326ed2c7f6748e2b98c1d7052427021cd9bb0168cb3d97294"} Jul 07 11:32:13.919426 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:13.918996 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gq77g" event={"ID":"87fd6ff0-9ee3-4be5-8f10-668b5e330994","Type":"ContainerStarted","Data":"724f39b5e9ac06f8de2098f33aa010e49d560cf20f3d2bd57ad4ccadc9d913f3"} Jul 07 11:32:13.943249 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:13.943197 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gq77g" podStartSLOduration=2.912210936 podStartE2EDuration="3.943180016s" totalImagesPullingTime="968.786136ms" totalInitContainerRuntime="62.182944ms" isStatefulPod=true podCreationTimestamp="2026-07-07 11:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:32:13.942525088 +0000 UTC m=+308.546861929" watchObservedRunningTime="2026-07-07 11:32:13.943180016 +0000 UTC m=+308.547516857" Jul 07 11:32:15.059345 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.059308 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7b45869b55-748t5"] Jul 07 11:32:15.062318 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.062301 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.065823 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.065801 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ea4agvjk3lr5\"" Jul 07 11:32:15.066016 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.065806 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 11:32:15.066016 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.065871 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 11:32:15.066016 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.065881 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 11:32:15.066016 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.065914 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-sfvtj\"" Jul 07 11:32:15.066016 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.065871 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 11:32:15.076655 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.076630 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b45869b55-748t5"] Jul 07 11:32:15.192657 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.192623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/302cdb48-badf-49ef-9c47-5c83ea33cd9b-client-ca-bundle\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.192865 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.192659 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/302cdb48-badf-49ef-9c47-5c83ea33cd9b-secret-metrics-server-tls\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.192865 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.192701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v625k\" (UniqueName: \"kubernetes.io/projected/302cdb48-badf-49ef-9c47-5c83ea33cd9b-kube-api-access-v625k\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.192865 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.192725 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/302cdb48-badf-49ef-9c47-5c83ea33cd9b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.192865 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.192779 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/302cdb48-badf-49ef-9c47-5c83ea33cd9b-secret-metrics-server-client-certs\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.192865 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.192819 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/302cdb48-badf-49ef-9c47-5c83ea33cd9b-metrics-server-audit-profiles\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.193051 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.192892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/302cdb48-badf-49ef-9c47-5c83ea33cd9b-audit-log\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.294136 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.294085 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v625k\" (UniqueName: \"kubernetes.io/projected/302cdb48-badf-49ef-9c47-5c83ea33cd9b-kube-api-access-v625k\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.294136 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.294141 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/302cdb48-badf-49ef-9c47-5c83ea33cd9b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.294476 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.294162 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/302cdb48-badf-49ef-9c47-5c83ea33cd9b-secret-metrics-server-client-certs\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.294476 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.294186 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/302cdb48-badf-49ef-9c47-5c83ea33cd9b-metrics-server-audit-profiles\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.294476 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.294407 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/302cdb48-badf-49ef-9c47-5c83ea33cd9b-audit-log\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.294476 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.294461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/302cdb48-badf-49ef-9c47-5c83ea33cd9b-client-ca-bundle\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.294766 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.294606 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/302cdb48-badf-49ef-9c47-5c83ea33cd9b-secret-metrics-server-tls\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.294824 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.294791 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/302cdb48-badf-49ef-9c47-5c83ea33cd9b-audit-log\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.295022 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.295003 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/302cdb48-badf-49ef-9c47-5c83ea33cd9b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.295211 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.295190 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/302cdb48-badf-49ef-9c47-5c83ea33cd9b-metrics-server-audit-profiles\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.296883 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.296861 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/302cdb48-badf-49ef-9c47-5c83ea33cd9b-secret-metrics-server-client-certs\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.297307 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.297286 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/302cdb48-badf-49ef-9c47-5c83ea33cd9b-secret-metrics-server-tls\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.297390 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.297311 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/302cdb48-badf-49ef-9c47-5c83ea33cd9b-client-ca-bundle\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.302230 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.302202 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v625k\" (UniqueName: \"kubernetes.io/projected/302cdb48-badf-49ef-9c47-5c83ea33cd9b-kube-api-access-v625k\") pod \"metrics-server-7b45869b55-748t5\" (UID: \"302cdb48-badf-49ef-9c47-5c83ea33cd9b\") " pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.371971 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.371878 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:15.504470 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.504428 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b45869b55-748t5"] Jul 07 11:32:15.507134 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:32:15.507106 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod302cdb48_badf_49ef_9c47_5c83ea33cd9b.slice/crio-38a8373055a9a27ae35100d79e2a030da613a720ea306ca1660a2a5c204a2a7a WatchSource:0}: Error finding container 38a8373055a9a27ae35100d79e2a030da613a720ea306ca1660a2a5c204a2a7a: Status 404 returned error can't find the container with id 38a8373055a9a27ae35100d79e2a030da613a720ea306ca1660a2a5c204a2a7a Jul 07 11:32:15.925982 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:15.925942 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b45869b55-748t5" event={"ID":"302cdb48-badf-49ef-9c47-5c83ea33cd9b","Type":"ContainerStarted","Data":"38a8373055a9a27ae35100d79e2a030da613a720ea306ca1660a2a5c204a2a7a"} Jul 07 11:32:16.695090 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.695054 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:32:16.698909 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.698886 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.701925 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.701900 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 11:32:16.704270 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.704247 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 11:32:16.704716 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.704693 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 11:32:16.706438 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.706296 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-lk7nv\"" Jul 07 11:32:16.708524 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.707259 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 11:32:16.708524 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.707410 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 11:32:16.708524 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.708421 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 11:32:16.709457 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.708960 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 11:32:16.709457 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.709106 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 11:32:16.709457 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.709415 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 11:32:16.710959 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.710938 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 11:32:16.711063 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.710960 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 11:32:16.711063 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.711012 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 11:32:16.711063 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.710942 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b71pnvs5kn2ht\"" Jul 07 11:32:16.713638 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.713619 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 11:32:16.714662 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.714644 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 11:32:16.726119 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.726086 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:32:16.808164 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808130 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-config\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808354 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808179 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808354 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808354 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808235 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808354 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808354 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808308 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808354 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-config-out\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808686 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808378 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808686 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808429 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808686 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808469 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808686 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808494 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808686 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808517 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808686 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808566 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808686 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808614 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wkmw\" (UniqueName: \"kubernetes.io/projected/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-kube-api-access-2wkmw\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.808686 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808673 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.809066 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.809066 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808752 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-web-config\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.809066 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808790 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.809066 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808828 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.809066 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.808852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910152 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910116 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910152 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910157 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910388 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910177 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910388 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-config\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910388 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910225 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910388 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910388 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910276 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910388 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910388 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910329 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910388 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-config-out\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910388 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910379 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910891 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910891 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910465 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910891 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910489 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910891 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910513 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910891 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910891 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910585 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wkmw\" (UniqueName: \"kubernetes.io/projected/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-kube-api-access-2wkmw\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910891 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910624 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910891 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910663 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.910891 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.910734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-web-config\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.911330 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.911056 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.911330 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.911144 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.911840 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.911809 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.913225 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.913132 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.913333 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.913244 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.913941 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.913698 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.914138 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.914106 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-config\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.914604 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.914581 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-web-config\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.914697 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.914656 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.915081 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.915056 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.915168 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.915078 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.915168 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.915120 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.915592 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.915551 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-config-out\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.915848 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.915825 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.916112 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.916088 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.916816 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.916776 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.916899 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.916877 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.917235 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.917208 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.917354 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.917280 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.921424 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.921397 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wkmw\" (UniqueName: \"kubernetes.io/projected/ad412cc7-72f2-4e70-a3d6-a6612bf07caa-kube-api-access-2wkmw\") pod \"prometheus-k8s-0\" (UID: \"ad412cc7-72f2-4e70-a3d6-a6612bf07caa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:16.932798 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:16.932772 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:32:17.011071 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:17.011035 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:17.154130 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:17.154084 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:17.154430 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:17.154402 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:17.160500 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:17.160469 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:17.161274 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:17.161250 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:32:17.164634 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:32:17.164610 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad412cc7_72f2_4e70_a3d6_a6612bf07caa.slice/crio-caa058ea895022b3448b12c18d96e97b913efddc1d34f94079a7edf48f3a899c WatchSource:0}: Error finding container caa058ea895022b3448b12c18d96e97b913efddc1d34f94079a7edf48f3a899c: Status 404 returned error can't find the container with id caa058ea895022b3448b12c18d96e97b913efddc1d34f94079a7edf48f3a899c Jul 07 11:32:17.467484 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:17.467431 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f7c65cbb6-28xbm"] Jul 07 11:32:17.937698 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:17.937617 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad412cc7-72f2-4e70-a3d6-a6612bf07caa","Type":"ContainerStarted","Data":"caa058ea895022b3448b12c18d96e97b913efddc1d34f94079a7edf48f3a899c"} Jul 07 11:32:17.938163 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:17.937705 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b45869b55-748t5" event={"ID":"302cdb48-badf-49ef-9c47-5c83ea33cd9b","Type":"ContainerStarted","Data":"8d74b9fb084f4e8aa412fccfeb70bca0c9176219f51d2bff63c37cd60a886282"} Jul 07 11:32:17.941387 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:17.941364 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:17.958576 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:17.958509 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7b45869b55-748t5" podStartSLOduration=1.468837443 podStartE2EDuration="2.95848814s" totalImagesPullingTime="1.489650697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:32:17.957650689 +0000 UTC m=+312.561987553" watchObservedRunningTime="2026-07-07 11:32:17.95848814 +0000 UTC m=+312.562824983" Jul 07 11:32:18.941936 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:18.941899 2578 generic.go:356] "Generic (PLEG): container finished" podID="ad412cc7-72f2-4e70-a3d6-a6612bf07caa" containerID="41afee29ff7c886d0e0a7f1073df1feb2d2c54cd2d8abf75826e885cfa61f6b2" exitCode=0 Jul 07 11:32:18.942470 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:18.941985 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad412cc7-72f2-4e70-a3d6-a6612bf07caa","Type":"ContainerDied","Data":"41afee29ff7c886d0e0a7f1073df1feb2d2c54cd2d8abf75826e885cfa61f6b2"} Jul 07 11:32:22.956926 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:22.956891 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad412cc7-72f2-4e70-a3d6-a6612bf07caa","Type":"ContainerStarted","Data":"2be79e284b826b04f44e9982a44ebdb90c1e14836973b9bd782b35cbded43854"} Jul 07 11:32:22.957327 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:22.956957 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad412cc7-72f2-4e70-a3d6-a6612bf07caa","Type":"ContainerStarted","Data":"551d558d15e275a94abc1c17ab9897d7a28d115fca2e36ffeb6fd3305f56a871"} Jul 07 11:32:24.966553 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:24.966512 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad412cc7-72f2-4e70-a3d6-a6612bf07caa","Type":"ContainerStarted","Data":"c5c5d28481a2ad6142b519d39182213d523bbf813f59aa687485c6e48eaf606a"} Jul 07 11:32:24.966907 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:24.966598 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad412cc7-72f2-4e70-a3d6-a6612bf07caa","Type":"ContainerStarted","Data":"cc180173f44119c8268a9ab7a583cb29fb101751510d810b01300628708b68cb"} Jul 07 11:32:24.966907 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:24.966624 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad412cc7-72f2-4e70-a3d6-a6612bf07caa","Type":"ContainerStarted","Data":"c6fba1a3efa9146d3c9aa77495e41c39ada0844db74e426ac6e639b48c515141"} Jul 07 11:32:24.966907 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:24.966653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad412cc7-72f2-4e70-a3d6-a6612bf07caa","Type":"ContainerStarted","Data":"d5dcaae3c9e5b49b78a1d1f30b0e5dabdc950e6d22f1c66c0471b09fac895602"} Jul 07 11:32:25.000283 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:25.000217 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.943775381 podStartE2EDuration="9.000192761s" totalImagesPullingTime="6.037415696s" totalInitContainerRuntime="19.001684ms" isStatefulPod=false podCreationTimestamp="2026-07-07 11:32:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:32:24.998404191 +0000 UTC m=+319.602741027" watchObservedRunningTime="2026-07-07 11:32:25.000192761 +0000 UTC m=+319.604529671" Jul 07 11:32:27.011860 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:27.011826 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:32:35.372247 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:35.372201 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:35.372816 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:35.372305 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:43.962005 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:43.961964 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7f7c65cbb6-28xbm" podUID="64854197-cf10-42f6-baea-2e0e5274b921" containerName="console" containerID="cri-o://48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584" gracePeriod=15 Jul 07 11:32:44.741768 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.741731 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:44.848824 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.848745 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-service-ca\") pod \"64854197-cf10-42f6-baea-2e0e5274b921\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " Jul 07 11:32:44.848824 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.848823 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/64854197-cf10-42f6-baea-2e0e5274b921-console-serving-cert\") pod \"64854197-cf10-42f6-baea-2e0e5274b921\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " Jul 07 11:32:44.849014 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.848922 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s87mw\" (UniqueName: \"kubernetes.io/projected/64854197-cf10-42f6-baea-2e0e5274b921-kube-api-access-s87mw\") pod \"64854197-cf10-42f6-baea-2e0e5274b921\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " Jul 07 11:32:44.849014 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.848954 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/64854197-cf10-42f6-baea-2e0e5274b921-console-oauth-config\") pod \"64854197-cf10-42f6-baea-2e0e5274b921\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " Jul 07 11:32:44.849014 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.848987 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-console-config\") pod \"64854197-cf10-42f6-baea-2e0e5274b921\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " Jul 07 11:32:44.849014 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.849011 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-trusted-ca-bundle\") pod \"64854197-cf10-42f6-baea-2e0e5274b921\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " Jul 07 11:32:44.849208 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.849050 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-oauth-serving-cert\") pod \"64854197-cf10-42f6-baea-2e0e5274b921\" (UID: \"64854197-cf10-42f6-baea-2e0e5274b921\") " Jul 07 11:32:44.849208 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.849165 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-service-ca" (OuterVolumeSpecName: "service-ca") pod "64854197-cf10-42f6-baea-2e0e5274b921" (UID: "64854197-cf10-42f6-baea-2e0e5274b921"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:32:44.849356 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.849330 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 11:32:44.849413 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.849355 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-console-config" (OuterVolumeSpecName: "console-config") pod "64854197-cf10-42f6-baea-2e0e5274b921" (UID: "64854197-cf10-42f6-baea-2e0e5274b921"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:32:44.849462 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.849436 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "64854197-cf10-42f6-baea-2e0e5274b921" (UID: "64854197-cf10-42f6-baea-2e0e5274b921"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:32:44.849679 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.849653 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "64854197-cf10-42f6-baea-2e0e5274b921" (UID: "64854197-cf10-42f6-baea-2e0e5274b921"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:32:44.851238 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.851211 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64854197-cf10-42f6-baea-2e0e5274b921-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "64854197-cf10-42f6-baea-2e0e5274b921" (UID: "64854197-cf10-42f6-baea-2e0e5274b921"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:32:44.851352 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.851241 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64854197-cf10-42f6-baea-2e0e5274b921-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "64854197-cf10-42f6-baea-2e0e5274b921" (UID: "64854197-cf10-42f6-baea-2e0e5274b921"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:32:44.851352 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.851295 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64854197-cf10-42f6-baea-2e0e5274b921-kube-api-access-s87mw" (OuterVolumeSpecName: "kube-api-access-s87mw") pod "64854197-cf10-42f6-baea-2e0e5274b921" (UID: "64854197-cf10-42f6-baea-2e0e5274b921"). InnerVolumeSpecName "kube-api-access-s87mw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:32:44.950670 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.950628 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/64854197-cf10-42f6-baea-2e0e5274b921-console-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 11:32:44.950670 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.950660 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s87mw\" (UniqueName: \"kubernetes.io/projected/64854197-cf10-42f6-baea-2e0e5274b921-kube-api-access-s87mw\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 11:32:44.950670 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.950675 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/64854197-cf10-42f6-baea-2e0e5274b921-console-oauth-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 11:32:44.950909 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.950686 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-console-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 11:32:44.950909 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.950697 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-trusted-ca-bundle\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 11:32:44.950909 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:44.950708 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/64854197-cf10-42f6-baea-2e0e5274b921-oauth-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 11:32:45.034992 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:45.034955 2578 generic.go:356] "Generic (PLEG): container finished" podID="64854197-cf10-42f6-baea-2e0e5274b921" containerID="48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584" exitCode=0 Jul 07 11:32:45.034992 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:45.034998 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7c65cbb6-28xbm" Jul 07 11:32:45.035526 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:45.035020 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7c65cbb6-28xbm" event={"ID":"64854197-cf10-42f6-baea-2e0e5274b921","Type":"ContainerDied","Data":"48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584"} Jul 07 11:32:45.035526 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:45.035069 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7c65cbb6-28xbm" event={"ID":"64854197-cf10-42f6-baea-2e0e5274b921","Type":"ContainerDied","Data":"3ba66d3a54a0a71c35002f08cccde4ac67fb3e14d52841f31a2fc8dd61deeaff"} Jul 07 11:32:45.035526 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:45.035096 2578 scope.go:117] "RemoveContainer" containerID="48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584" Jul 07 11:32:45.043670 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:45.043648 2578 scope.go:117] "RemoveContainer" containerID="48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584" Jul 07 11:32:45.043986 ip-10-0-139-127 kubenswrapper[2578]: E0707 11:32:45.043946 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584\": container with ID starting with 48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584 not found: ID does not exist" containerID="48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584" Jul 07 11:32:45.044042 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:45.043993 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584"} err="failed to get container status \"48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584\": rpc error: code = NotFound desc = could not find container \"48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584\": container with ID starting with 48964c4c0f7790db2e42ae90abb2a08c8bf3dd585c932c281e5ffc890468a584 not found: ID does not exist" Jul 07 11:32:45.055016 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:45.054984 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f7c65cbb6-28xbm"] Jul 07 11:32:45.058619 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:45.058594 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f7c65cbb6-28xbm"] Jul 07 11:32:45.937889 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:45.937853 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64854197-cf10-42f6-baea-2e0e5274b921" path="/var/lib/kubelet/pods/64854197-cf10-42f6-baea-2e0e5274b921/volumes" Jul 07 11:32:48.786929 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:48.786887 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" podUID="65910b63-9b97-49ae-b758-ff9de6cc7a52" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 07 11:32:55.378162 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:55.378132 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:55.382048 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:55.382022 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7b45869b55-748t5" Jul 07 11:32:58.787904 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:32:58.787858 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" podUID="65910b63-9b97-49ae-b758-ff9de6cc7a52" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 07 11:33:08.787412 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:08.787370 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" podUID="65910b63-9b97-49ae-b758-ff9de6cc7a52" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 07 11:33:08.787886 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:08.787453 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" Jul 07 11:33:08.787955 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:08.787937 2578 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"ef9a9640abe5112772b19d9ec12a0701926026ccb0f02d070e8280a355fe82d6"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 07 11:33:08.787997 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:08.787973 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" podUID="65910b63-9b97-49ae-b758-ff9de6cc7a52" containerName="service-proxy" containerID="cri-o://ef9a9640abe5112772b19d9ec12a0701926026ccb0f02d070e8280a355fe82d6" gracePeriod=30 Jul 07 11:33:09.122509 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:09.122432 2578 generic.go:356] "Generic (PLEG): container finished" podID="65910b63-9b97-49ae-b758-ff9de6cc7a52" containerID="ef9a9640abe5112772b19d9ec12a0701926026ccb0f02d070e8280a355fe82d6" exitCode=2 Jul 07 11:33:09.122666 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:09.122507 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" event={"ID":"65910b63-9b97-49ae-b758-ff9de6cc7a52","Type":"ContainerDied","Data":"ef9a9640abe5112772b19d9ec12a0701926026ccb0f02d070e8280a355fe82d6"} Jul 07 11:33:09.122666 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:09.122568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564dc7fc85-xgkl6" event={"ID":"65910b63-9b97-49ae-b758-ff9de6cc7a52","Type":"ContainerStarted","Data":"27d8b0d503fec714c6019e56d73425f1640fedc427b98b053fdee3c4e3d31b7b"} Jul 07 11:33:17.011277 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:17.011241 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:33:17.039481 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:17.039444 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:33:17.169186 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:17.169147 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:33:19.741477 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:19.741440 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:33:19.743815 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:19.743789 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e825e10d-32d1-41d4-aa12-1d1f1dbbb429-metrics-certs\") pod \"network-metrics-daemon-9vm97\" (UID: \"e825e10d-32d1-41d4-aa12-1d1f1dbbb429\") " pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:33:19.935974 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:19.935946 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nfmsb\"" Jul 07 11:33:19.944323 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:19.944296 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vm97" Jul 07 11:33:20.070523 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:20.070488 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9vm97"] Jul 07 11:33:20.073426 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:33:20.073397 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode825e10d_32d1_41d4_aa12_1d1f1dbbb429.slice/crio-e6aaa528d0ed46bed19664570b7c31794296649f3e29b2bef03b249fa30f4837 WatchSource:0}: Error finding container e6aaa528d0ed46bed19664570b7c31794296649f3e29b2bef03b249fa30f4837: Status 404 returned error can't find the container with id e6aaa528d0ed46bed19664570b7c31794296649f3e29b2bef03b249fa30f4837 Jul 07 11:33:20.162949 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:20.162912 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9vm97" event={"ID":"e825e10d-32d1-41d4-aa12-1d1f1dbbb429","Type":"ContainerStarted","Data":"e6aaa528d0ed46bed19664570b7c31794296649f3e29b2bef03b249fa30f4837"} Jul 07 11:33:21.168439 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:21.168413 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9vm97" event={"ID":"e825e10d-32d1-41d4-aa12-1d1f1dbbb429","Type":"ContainerStarted","Data":"445f3f98a6c32850cffad6cab3d71f4b33bb0f6ad36fbf4c9b782a9892d19a70"} Jul 07 11:33:22.173158 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:22.173123 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9vm97" event={"ID":"e825e10d-32d1-41d4-aa12-1d1f1dbbb429","Type":"ContainerStarted","Data":"e3e9b11cbe5dd4a057e1b120d7009a2a3ab2aeaa8483b9094e8046b07a117ab9"} Jul 07 11:33:22.191792 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:33:22.191733 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9vm97" podStartSLOduration=375.20220711 podStartE2EDuration="6m16.191713511s" totalImagesPullingTime="989.506401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:33:22.190193449 +0000 UTC m=+376.794530292" watchObservedRunningTime="2026-07-07 11:33:22.191713511 +0000 UTC m=+376.796050376" Jul 07 11:36:12.186115 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.186084 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k"] Jul 07 11:36:12.186512 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.186358 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64854197-cf10-42f6-baea-2e0e5274b921" containerName="console" Jul 07 11:36:12.186512 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.186369 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="64854197-cf10-42f6-baea-2e0e5274b921" containerName="console" Jul 07 11:36:12.186512 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.186413 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="64854197-cf10-42f6-baea-2e0e5274b921" containerName="console" Jul 07 11:36:12.189303 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.189276 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.191800 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.191771 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 11:36:12.191800 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.191781 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-29df8\"" Jul 07 11:36:12.192257 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.192239 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 11:36:12.192463 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.192446 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 11:36:12.192555 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.192498 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 11:36:12.222252 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.222222 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k"] Jul 07 11:36:12.239917 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.239889 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c48f684-542f-438a-91a7-d924a8027930-tmp\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.240066 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.239927 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c48f684-542f-438a-91a7-d924a8027930-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.240066 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.239957 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c48f684-542f-438a-91a7-d924a8027930-webhook-cert\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.240066 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.240020 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmnmm\" (UniqueName: \"kubernetes.io/projected/6c48f684-542f-438a-91a7-d924a8027930-kube-api-access-wmnmm\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.240066 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.240045 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c48f684-542f-438a-91a7-d924a8027930-manifests\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.340749 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.340712 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c48f684-542f-438a-91a7-d924a8027930-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.340928 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.340758 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c48f684-542f-438a-91a7-d924a8027930-webhook-cert\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.340997 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.340947 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmnmm\" (UniqueName: \"kubernetes.io/projected/6c48f684-542f-438a-91a7-d924a8027930-kube-api-access-wmnmm\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.341052 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.341003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c48f684-542f-438a-91a7-d924a8027930-manifests\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.341101 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.341071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c48f684-542f-438a-91a7-d924a8027930-tmp\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.341350 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.341328 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c48f684-542f-438a-91a7-d924a8027930-manifests\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.343107 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.343077 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c48f684-542f-438a-91a7-d924a8027930-tmp\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.343269 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.343252 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c48f684-542f-438a-91a7-d924a8027930-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.343330 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.343298 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c48f684-542f-438a-91a7-d924a8027930-webhook-cert\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.354784 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.354759 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmnmm\" (UniqueName: \"kubernetes.io/projected/6c48f684-542f-438a-91a7-d924a8027930-kube-api-access-wmnmm\") pod \"opendatahub-operator-controller-manager-f68d9489b-gfv5k\" (UID: \"6c48f684-542f-438a-91a7-d924a8027930\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.500336 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.500254 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:12.632084 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.632062 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k"] Jul 07 11:36:12.634722 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:36:12.634699 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c48f684_542f_438a_91a7_d924a8027930.slice/crio-58257a9bf7c58219a5f0f74741ad794f47af046d3da781d470cb62dd1906e170 WatchSource:0}: Error finding container 58257a9bf7c58219a5f0f74741ad794f47af046d3da781d470cb62dd1906e170: Status 404 returned error can't find the container with id 58257a9bf7c58219a5f0f74741ad794f47af046d3da781d470cb62dd1906e170 Jul 07 11:36:12.753803 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:12.753716 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" event={"ID":"6c48f684-542f-438a-91a7-d924a8027930","Type":"ContainerStarted","Data":"58257a9bf7c58219a5f0f74741ad794f47af046d3da781d470cb62dd1906e170"} Jul 07 11:36:15.767158 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:15.767121 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" event={"ID":"6c48f684-542f-438a-91a7-d924a8027930","Type":"ContainerStarted","Data":"de5980728bda4b431a783ac219f67cafa8ba1df14dcbfe7d2094d98f89ecac9f"} Jul 07 11:36:16.773305 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:16.773273 2578 generic.go:356] "Generic (PLEG): container finished" podID="6c48f684-542f-438a-91a7-d924a8027930" containerID="de5980728bda4b431a783ac219f67cafa8ba1df14dcbfe7d2094d98f89ecac9f" exitCode=0 Jul 07 11:36:16.773716 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:16.773314 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" event={"ID":"6c48f684-542f-438a-91a7-d924a8027930","Type":"ContainerDied","Data":"de5980728bda4b431a783ac219f67cafa8ba1df14dcbfe7d2094d98f89ecac9f"} Jul 07 11:36:17.780097 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:17.779996 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" event={"ID":"6c48f684-542f-438a-91a7-d924a8027930","Type":"ContainerStarted","Data":"b5e9d1036a8305ee0190013a391c82900db14a14b357944a4dded82928b85401"} Jul 07 11:36:17.780587 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:17.780158 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:17.804392 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:17.804340 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" podStartSLOduration=1.105501678 podStartE2EDuration="5.804323386s" totalImagesPullingTime="3.616330773s" totalInitContainerRuntime="1.082490935s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:36:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:36:17.8029397 +0000 UTC m=+552.407276554" watchObservedRunningTime="2026-07-07 11:36:17.804323386 +0000 UTC m=+552.408660227" Jul 07 11:36:28.785681 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:28.785652 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-gfv5k" Jul 07 11:36:39.615791 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.615761 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9"] Jul 07 11:36:39.620916 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.620898 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.624926 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.624898 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 11:36:39.624926 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.624916 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 11:36:39.625104 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.624951 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7sw6n\"" Jul 07 11:36:39.625104 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.624955 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 11:36:39.625104 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.624973 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 11:36:39.625104 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.624993 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 11:36:39.625350 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.625336 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 11:36:39.632323 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.632295 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9"] Jul 07 11:36:39.767145 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.767117 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e034319-6cc8-44a2-b487-b6e05a3927b6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.767309 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.767147 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e034319-6cc8-44a2-b487-b6e05a3927b6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.767309 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.767181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e034319-6cc8-44a2-b487-b6e05a3927b6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.767309 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.767267 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e034319-6cc8-44a2-b487-b6e05a3927b6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.767309 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.767299 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtdps\" (UniqueName: \"kubernetes.io/projected/3e034319-6cc8-44a2-b487-b6e05a3927b6-kube-api-access-gtdps\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.767443 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.767338 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e034319-6cc8-44a2-b487-b6e05a3927b6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.767443 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.767355 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e034319-6cc8-44a2-b487-b6e05a3927b6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.867966 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.867890 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e034319-6cc8-44a2-b487-b6e05a3927b6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.867966 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.867921 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e034319-6cc8-44a2-b487-b6e05a3927b6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.867966 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.867945 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e034319-6cc8-44a2-b487-b6e05a3927b6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.868196 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.867971 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e034319-6cc8-44a2-b487-b6e05a3927b6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.868196 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.868091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtdps\" (UniqueName: \"kubernetes.io/projected/3e034319-6cc8-44a2-b487-b6e05a3927b6-kube-api-access-gtdps\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.868196 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.868145 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e034319-6cc8-44a2-b487-b6e05a3927b6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.868348 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.868285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e034319-6cc8-44a2-b487-b6e05a3927b6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.868705 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.868684 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e034319-6cc8-44a2-b487-b6e05a3927b6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.870241 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.870221 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e034319-6cc8-44a2-b487-b6e05a3927b6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.870412 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.870391 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e034319-6cc8-44a2-b487-b6e05a3927b6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.870527 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.870509 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e034319-6cc8-44a2-b487-b6e05a3927b6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.870614 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.870513 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e034319-6cc8-44a2-b487-b6e05a3927b6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.880387 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.880364 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtdps\" (UniqueName: \"kubernetes.io/projected/3e034319-6cc8-44a2-b487-b6e05a3927b6-kube-api-access-gtdps\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.880630 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.880611 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e034319-6cc8-44a2-b487-b6e05a3927b6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-b78z9\" (UID: \"3e034319-6cc8-44a2-b487-b6e05a3927b6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:39.929398 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:39.929373 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:40.084088 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:40.083905 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9"] Jul 07 11:36:40.086715 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:36:40.086681 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e034319_6cc8_44a2_b487_b6e05a3927b6.slice/crio-e075cec2911c6b160ce639016de74f5f73b315a2d9fc9576c11df3723cf3b83c WatchSource:0}: Error finding container e075cec2911c6b160ce639016de74f5f73b315a2d9fc9576c11df3723cf3b83c: Status 404 returned error can't find the container with id e075cec2911c6b160ce639016de74f5f73b315a2d9fc9576c11df3723cf3b83c Jul 07 11:36:40.866374 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:40.866269 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" event={"ID":"3e034319-6cc8-44a2-b487-b6e05a3927b6","Type":"ContainerStarted","Data":"e075cec2911c6b160ce639016de74f5f73b315a2d9fc9576c11df3723cf3b83c"} Jul 07 11:36:42.085327 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.085295 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27"] Jul 07 11:36:42.090342 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.090321 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.102968 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.102943 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 11:36:42.103314 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.103119 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 11:36:42.103314 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.103215 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:36:42.104630 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.104610 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 11:36:42.104739 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.104709 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-m9542\"" Jul 07 11:36:42.107254 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.107237 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 11:36:42.132169 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.132103 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27"] Jul 07 11:36:42.188231 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.188199 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/594d0f69-d382-42e6-aab9-e2d7732622b5-manager-config\") pod \"lws-controller-manager-69986dd77b-6tp27\" (UID: \"594d0f69-d382-42e6-aab9-e2d7732622b5\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.188405 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.188296 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxnrq\" (UniqueName: \"kubernetes.io/projected/594d0f69-d382-42e6-aab9-e2d7732622b5-kube-api-access-sxnrq\") pod \"lws-controller-manager-69986dd77b-6tp27\" (UID: \"594d0f69-d382-42e6-aab9-e2d7732622b5\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.188405 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.188323 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/594d0f69-d382-42e6-aab9-e2d7732622b5-cert\") pod \"lws-controller-manager-69986dd77b-6tp27\" (UID: \"594d0f69-d382-42e6-aab9-e2d7732622b5\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.188405 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.188340 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/594d0f69-d382-42e6-aab9-e2d7732622b5-metrics-cert\") pod \"lws-controller-manager-69986dd77b-6tp27\" (UID: \"594d0f69-d382-42e6-aab9-e2d7732622b5\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.289484 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.289447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/594d0f69-d382-42e6-aab9-e2d7732622b5-manager-config\") pod \"lws-controller-manager-69986dd77b-6tp27\" (UID: \"594d0f69-d382-42e6-aab9-e2d7732622b5\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.289691 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.289585 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxnrq\" (UniqueName: \"kubernetes.io/projected/594d0f69-d382-42e6-aab9-e2d7732622b5-kube-api-access-sxnrq\") pod \"lws-controller-manager-69986dd77b-6tp27\" (UID: \"594d0f69-d382-42e6-aab9-e2d7732622b5\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.289691 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.289620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/594d0f69-d382-42e6-aab9-e2d7732622b5-cert\") pod \"lws-controller-manager-69986dd77b-6tp27\" (UID: \"594d0f69-d382-42e6-aab9-e2d7732622b5\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.289691 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.289645 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/594d0f69-d382-42e6-aab9-e2d7732622b5-metrics-cert\") pod \"lws-controller-manager-69986dd77b-6tp27\" (UID: \"594d0f69-d382-42e6-aab9-e2d7732622b5\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.290075 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.290046 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/594d0f69-d382-42e6-aab9-e2d7732622b5-manager-config\") pod \"lws-controller-manager-69986dd77b-6tp27\" (UID: \"594d0f69-d382-42e6-aab9-e2d7732622b5\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.291985 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.291959 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/594d0f69-d382-42e6-aab9-e2d7732622b5-cert\") pod \"lws-controller-manager-69986dd77b-6tp27\" (UID: \"594d0f69-d382-42e6-aab9-e2d7732622b5\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.292144 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.292123 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/594d0f69-d382-42e6-aab9-e2d7732622b5-metrics-cert\") pod \"lws-controller-manager-69986dd77b-6tp27\" (UID: \"594d0f69-d382-42e6-aab9-e2d7732622b5\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.303784 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.303761 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxnrq\" (UniqueName: \"kubernetes.io/projected/594d0f69-d382-42e6-aab9-e2d7732622b5-kube-api-access-sxnrq\") pod \"lws-controller-manager-69986dd77b-6tp27\" (UID: \"594d0f69-d382-42e6-aab9-e2d7732622b5\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.320519 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.320483 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 11:36:42.320628 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.320568 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 11:36:42.401246 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.401220 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:42.555072 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.555013 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27"] Jul 07 11:36:42.557179 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:36:42.557148 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod594d0f69_d382_42e6_aab9_e2d7732622b5.slice/crio-d87b3ed16a547b845c2ce013608b455e8a0b1bba709e00c0c98d6d36fe18f83b WatchSource:0}: Error finding container d87b3ed16a547b845c2ce013608b455e8a0b1bba709e00c0c98d6d36fe18f83b: Status 404 returned error can't find the container with id d87b3ed16a547b845c2ce013608b455e8a0b1bba709e00c0c98d6d36fe18f83b Jul 07 11:36:42.875954 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.875915 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" event={"ID":"594d0f69-d382-42e6-aab9-e2d7732622b5","Type":"ContainerStarted","Data":"d87b3ed16a547b845c2ce013608b455e8a0b1bba709e00c0c98d6d36fe18f83b"} Jul 07 11:36:42.877873 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.877844 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" event={"ID":"3e034319-6cc8-44a2-b487-b6e05a3927b6","Type":"ContainerStarted","Data":"1bc2bc5d97b12ab7486ec49f0cc5935bf5275b0b2978ed38ddd279cf86cd34ed"} Jul 07 11:36:42.878015 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.878003 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:42.917773 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:42.917717 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" podStartSLOduration=1.686356454 podStartE2EDuration="3.917695884s" totalImagesPullingTime="2.23133943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:36:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:36:42.916656707 +0000 UTC m=+577.520993548" watchObservedRunningTime="2026-07-07 11:36:42.917695884 +0000 UTC m=+577.522032726" Jul 07 11:36:43.884455 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:43.884425 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b78z9" Jul 07 11:36:45.892065 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:45.892032 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" event={"ID":"594d0f69-d382-42e6-aab9-e2d7732622b5","Type":"ContainerStarted","Data":"c9353000a95b83af097fc55b06b16d441cfadc2d50bc09a2d8e91082078e246d"} Jul 07 11:36:45.892608 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:45.892134 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:36:45.961446 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:45.961385 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" podStartSLOduration=1.528488149 podStartE2EDuration="3.961370588s" totalImagesPullingTime="2.432882439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:36:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:36:45.960460369 +0000 UTC m=+580.564797211" watchObservedRunningTime="2026-07-07 11:36:45.961370588 +0000 UTC m=+580.565707429" Jul 07 11:36:56.900617 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:36:56.900585 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-6tp27" Jul 07 11:37:05.994610 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:37:05.994480 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:37:05.999286 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:37:05.995116 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:38:49.482482 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.482445 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bb858c6d6-zjt42"] Jul 07 11:38:49.484509 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.484483 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.486932 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.486910 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 11:38:49.487934 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.487914 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 11:38:49.488044 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.487956 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 11:38:49.488044 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.488026 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 11:38:49.488270 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.488254 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mjmt6\"" Jul 07 11:38:49.488340 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.488306 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 11:38:49.488340 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.488325 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 11:38:49.488442 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.488367 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 11:38:49.492528 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.492511 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 11:38:49.500385 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.500360 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bb858c6d6-zjt42"] Jul 07 11:38:49.574115 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.574079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54de5cb3-dd83-4415-be7e-711c40fe259d-oauth-serving-cert\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.574284 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.574132 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54de5cb3-dd83-4415-be7e-711c40fe259d-trusted-ca-bundle\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.574284 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.574186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54de5cb3-dd83-4415-be7e-711c40fe259d-console-oauth-config\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.574284 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.574233 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54de5cb3-dd83-4415-be7e-711c40fe259d-service-ca\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.574284 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.574271 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54de5cb3-dd83-4415-be7e-711c40fe259d-console-serving-cert\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.574420 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.574291 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl7tx\" (UniqueName: \"kubernetes.io/projected/54de5cb3-dd83-4415-be7e-711c40fe259d-kube-api-access-rl7tx\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.574420 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.574307 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54de5cb3-dd83-4415-be7e-711c40fe259d-console-config\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.675531 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.675501 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54de5cb3-dd83-4415-be7e-711c40fe259d-trusted-ca-bundle\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.675739 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.675548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54de5cb3-dd83-4415-be7e-711c40fe259d-console-oauth-config\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.675739 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.675572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54de5cb3-dd83-4415-be7e-711c40fe259d-service-ca\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.675739 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.675608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54de5cb3-dd83-4415-be7e-711c40fe259d-console-serving-cert\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.675739 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.675636 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl7tx\" (UniqueName: \"kubernetes.io/projected/54de5cb3-dd83-4415-be7e-711c40fe259d-kube-api-access-rl7tx\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.675739 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.675662 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54de5cb3-dd83-4415-be7e-711c40fe259d-console-config\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.675739 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.675711 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54de5cb3-dd83-4415-be7e-711c40fe259d-oauth-serving-cert\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.676477 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.676444 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54de5cb3-dd83-4415-be7e-711c40fe259d-oauth-serving-cert\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.676614 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.676490 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54de5cb3-dd83-4415-be7e-711c40fe259d-console-config\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.676614 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.676492 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54de5cb3-dd83-4415-be7e-711c40fe259d-trusted-ca-bundle\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.676614 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.676553 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54de5cb3-dd83-4415-be7e-711c40fe259d-service-ca\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.678124 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.678102 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54de5cb3-dd83-4415-be7e-711c40fe259d-console-oauth-config\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.678638 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.678617 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54de5cb3-dd83-4415-be7e-711c40fe259d-console-serving-cert\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.684680 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.684657 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl7tx\" (UniqueName: \"kubernetes.io/projected/54de5cb3-dd83-4415-be7e-711c40fe259d-kube-api-access-rl7tx\") pod \"console-6bb858c6d6-zjt42\" (UID: \"54de5cb3-dd83-4415-be7e-711c40fe259d\") " pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.793808 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.793720 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:49.919642 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:49.919620 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bb858c6d6-zjt42"] Jul 07 11:38:49.922278 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:38:49.922246 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54de5cb3_dd83_4415_be7e_711c40fe259d.slice/crio-8019ca5169c0397df2ff0224ae851a96c8b78ba48308adc8bec08f1a6e008133 WatchSource:0}: Error finding container 8019ca5169c0397df2ff0224ae851a96c8b78ba48308adc8bec08f1a6e008133: Status 404 returned error can't find the container with id 8019ca5169c0397df2ff0224ae851a96c8b78ba48308adc8bec08f1a6e008133 Jul 07 11:38:50.342146 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:50.342108 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb858c6d6-zjt42" event={"ID":"54de5cb3-dd83-4415-be7e-711c40fe259d","Type":"ContainerStarted","Data":"9d31914ab00305e7dc18b3d77b06fe86b005910dd15fe8b17b1202334c0048f2"} Jul 07 11:38:50.342348 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:50.342159 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb858c6d6-zjt42" event={"ID":"54de5cb3-dd83-4415-be7e-711c40fe259d","Type":"ContainerStarted","Data":"8019ca5169c0397df2ff0224ae851a96c8b78ba48308adc8bec08f1a6e008133"} Jul 07 11:38:50.362900 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:50.362845 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bb858c6d6-zjt42" podStartSLOduration=1.362827138 podStartE2EDuration="1.362827138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:38:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:38:50.361370635 +0000 UTC m=+704.965707518" watchObservedRunningTime="2026-07-07 11:38:50.362827138 +0000 UTC m=+704.967163978" Jul 07 11:38:59.794825 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:59.794784 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:59.795349 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:59.794864 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:38:59.799490 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:38:59.799468 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:39:00.383247 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:00.383213 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bb858c6d6-zjt42" Jul 07 11:39:26.477293 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.477263 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:39:26.479345 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.479329 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" Jul 07 11:39:26.482931 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.482907 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 11:39:26.483041 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.483018 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 11:39:26.483094 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.483080 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2cjpz\"" Jul 07 11:39:26.483745 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.483733 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 11:39:26.492937 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.492916 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:39:26.548010 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.547982 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:39:26.594082 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.594055 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56rbw\" (UniqueName: \"kubernetes.io/projected/17ac8a3e-a3af-47f6-b21e-9935be5f72d1-kube-api-access-56rbw\") pod \"limitador-limitador-54d6d988d5-xtmx4\" (UID: \"17ac8a3e-a3af-47f6-b21e-9935be5f72d1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" Jul 07 11:39:26.594248 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.594115 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17ac8a3e-a3af-47f6-b21e-9935be5f72d1-config-file\") pod \"limitador-limitador-54d6d988d5-xtmx4\" (UID: \"17ac8a3e-a3af-47f6-b21e-9935be5f72d1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" Jul 07 11:39:26.695408 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.695377 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56rbw\" (UniqueName: \"kubernetes.io/projected/17ac8a3e-a3af-47f6-b21e-9935be5f72d1-kube-api-access-56rbw\") pod \"limitador-limitador-54d6d988d5-xtmx4\" (UID: \"17ac8a3e-a3af-47f6-b21e-9935be5f72d1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" Jul 07 11:39:26.695572 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.695422 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17ac8a3e-a3af-47f6-b21e-9935be5f72d1-config-file\") pod \"limitador-limitador-54d6d988d5-xtmx4\" (UID: \"17ac8a3e-a3af-47f6-b21e-9935be5f72d1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" Jul 07 11:39:26.696086 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.696067 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17ac8a3e-a3af-47f6-b21e-9935be5f72d1-config-file\") pod \"limitador-limitador-54d6d988d5-xtmx4\" (UID: \"17ac8a3e-a3af-47f6-b21e-9935be5f72d1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" Jul 07 11:39:26.704493 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.704470 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56rbw\" (UniqueName: \"kubernetes.io/projected/17ac8a3e-a3af-47f6-b21e-9935be5f72d1-kube-api-access-56rbw\") pod \"limitador-limitador-54d6d988d5-xtmx4\" (UID: \"17ac8a3e-a3af-47f6-b21e-9935be5f72d1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" Jul 07 11:39:26.789135 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.789052 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" Jul 07 11:39:26.915113 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.915084 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:39:26.918187 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:39:26.918151 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ac8a3e_a3af_47f6_b21e_9935be5f72d1.slice/crio-f4f5feb88ab50011eaeb0ed68d1fb2e7ef45935128d9dfff6dc1d6091df8a137 WatchSource:0}: Error finding container f4f5feb88ab50011eaeb0ed68d1fb2e7ef45935128d9dfff6dc1d6091df8a137: Status 404 returned error can't find the container with id f4f5feb88ab50011eaeb0ed68d1fb2e7ef45935128d9dfff6dc1d6091df8a137 Jul 07 11:39:26.920063 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:26.920046 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:39:27.484221 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:27.484140 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" event={"ID":"17ac8a3e-a3af-47f6-b21e-9935be5f72d1","Type":"ContainerStarted","Data":"f4f5feb88ab50011eaeb0ed68d1fb2e7ef45935128d9dfff6dc1d6091df8a137"} Jul 07 11:39:29.493722 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:29.493688 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" event={"ID":"17ac8a3e-a3af-47f6-b21e-9935be5f72d1","Type":"ContainerStarted","Data":"5ea907812ed6eba34aba30470f34123159a43fed049f3cf9b7700ae5c752feb2"} Jul 07 11:39:29.494110 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:29.493816 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" Jul 07 11:39:40.498951 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:40.498923 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" Jul 07 11:39:40.516957 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:39:40.516894 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-xtmx4" podStartSLOduration=12.169339145 podStartE2EDuration="14.516878801s" totalImagesPullingTime="2.347539656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:39:29.51214368 +0000 UTC m=+744.116480533" watchObservedRunningTime="2026-07-07 11:39:40.516878801 +0000 UTC m=+755.121215642" Jul 07 11:40:37.623732 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:40:37.623658 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:41:00.225756 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.225728 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz"] Jul 07 11:41:00.228786 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.228766 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.232799 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.232775 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9wjpf\"" Jul 07 11:41:00.232910 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.232819 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 11:41:00.232910 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.232837 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 11:41:00.232910 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.232841 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 11:41:00.242259 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.242231 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz"] Jul 07 11:41:00.382751 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.382720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.382751 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.382757 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.382991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.382825 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdc4s\" (UniqueName: \"kubernetes.io/projected/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-kube-api-access-fdc4s\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.382991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.382863 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.382991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.382882 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.382991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.382899 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.483396 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.483318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdc4s\" (UniqueName: \"kubernetes.io/projected/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-kube-api-access-fdc4s\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.483396 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.483358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.483396 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.483381 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.483654 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.483407 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.483654 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.483446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.483654 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.483477 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.483797 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.483772 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.483852 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.483833 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.483930 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.483909 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.485779 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.485757 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.485998 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.485980 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.491277 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.491247 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdc4s\" (UniqueName: \"kubernetes.io/projected/0a95a75e-3af2-4180-a0c5-c24b9b9a11f1-kube-api-access-fdc4s\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz\" (UID: \"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.539052 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.539025 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:00.677045 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.677020 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz"] Jul 07 11:41:00.679387 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:41:00.679358 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a95a75e_3af2_4180_a0c5_c24b9b9a11f1.slice/crio-d46be6b85931cb433d398b2011d3cc6ea68f204dfb54b3564b1c2e946905faf8 WatchSource:0}: Error finding container d46be6b85931cb433d398b2011d3cc6ea68f204dfb54b3564b1c2e946905faf8: Status 404 returned error can't find the container with id d46be6b85931cb433d398b2011d3cc6ea68f204dfb54b3564b1c2e946905faf8 Jul 07 11:41:00.720113 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.720079 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:41:00.835188 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:00.835109 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" event={"ID":"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1","Type":"ContainerStarted","Data":"d46be6b85931cb433d398b2011d3cc6ea68f204dfb54b3564b1c2e946905faf8"} Jul 07 11:41:02.846222 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:02.846186 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" event={"ID":"0a95a75e-3af2-4180-a0c5-c24b9b9a11f1","Type":"ContainerStarted","Data":"e1ebd8379056b674ecf09157924d18ea0ac5a3ae3dd4edd1b69428b06169d052"} Jul 07 11:41:02.865043 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:02.864904 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" podStartSLOduration=1.748640067 podStartE2EDuration="2.864884981s" totalImagesPullingTime="1.116244914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:02.86457732 +0000 UTC m=+837.468914162" watchObservedRunningTime="2026-07-07 11:41:02.864884981 +0000 UTC m=+837.469221823" Jul 07 11:41:04.524011 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:04.523976 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:41:10.539676 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:10.539625 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:10.540199 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:10.539699 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:10.552137 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:10.552097 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:10.890790 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:10.890712 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz" Jul 07 11:41:21.611941 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:21.611905 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:41:30.729883 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:30.729853 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:41:36.413948 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.413915 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq"] Jul 07 11:41:36.417098 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.417080 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.419591 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.419566 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 11:41:36.429238 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.429210 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq"] Jul 07 11:41:36.491508 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.491468 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vz5l\" (UniqueName: \"kubernetes.io/projected/60f314e0-3447-4a36-bf24-6dfce019b8fc-kube-api-access-8vz5l\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.491694 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.491520 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60f314e0-3447-4a36-bf24-6dfce019b8fc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.491694 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.491618 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60f314e0-3447-4a36-bf24-6dfce019b8fc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.491694 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.491679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60f314e0-3447-4a36-bf24-6dfce019b8fc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.491805 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.491698 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60f314e0-3447-4a36-bf24-6dfce019b8fc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.491805 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.491721 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60f314e0-3447-4a36-bf24-6dfce019b8fc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.592821 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.592780 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60f314e0-3447-4a36-bf24-6dfce019b8fc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.592991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.592848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60f314e0-3447-4a36-bf24-6dfce019b8fc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.592991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.592875 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60f314e0-3447-4a36-bf24-6dfce019b8fc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.592991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.592896 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60f314e0-3447-4a36-bf24-6dfce019b8fc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.592991 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.592922 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vz5l\" (UniqueName: \"kubernetes.io/projected/60f314e0-3447-4a36-bf24-6dfce019b8fc-kube-api-access-8vz5l\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.593199 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.593072 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60f314e0-3447-4a36-bf24-6dfce019b8fc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.593287 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.593265 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60f314e0-3447-4a36-bf24-6dfce019b8fc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.593347 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.593274 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60f314e0-3447-4a36-bf24-6dfce019b8fc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.593347 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.593329 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60f314e0-3447-4a36-bf24-6dfce019b8fc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.595237 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.595214 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60f314e0-3447-4a36-bf24-6dfce019b8fc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.595381 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.595363 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60f314e0-3447-4a36-bf24-6dfce019b8fc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.602046 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.602019 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vz5l\" (UniqueName: \"kubernetes.io/projected/60f314e0-3447-4a36-bf24-6dfce019b8fc-kube-api-access-8vz5l\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6mkmq\" (UID: \"60f314e0-3447-4a36-bf24-6dfce019b8fc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.728359 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.728276 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:36.856494 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.856462 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq"] Jul 07 11:41:36.859478 ip-10-0-139-127 kubenswrapper[2578]: W0707 11:41:36.859452 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60f314e0_3447_4a36_bf24_6dfce019b8fc.slice/crio-a3bf4f614f49524c0ee13be1b08e84518acf542b164dbf40514d7eb16fabb7f5 WatchSource:0}: Error finding container a3bf4f614f49524c0ee13be1b08e84518acf542b164dbf40514d7eb16fabb7f5: Status 404 returned error can't find the container with id a3bf4f614f49524c0ee13be1b08e84518acf542b164dbf40514d7eb16fabb7f5 Jul 07 11:41:36.983335 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:36.983254 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" event={"ID":"60f314e0-3447-4a36-bf24-6dfce019b8fc","Type":"ContainerStarted","Data":"a3bf4f614f49524c0ee13be1b08e84518acf542b164dbf40514d7eb16fabb7f5"} Jul 07 11:41:37.988849 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:37.988818 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" event={"ID":"60f314e0-3447-4a36-bf24-6dfce019b8fc","Type":"ContainerStarted","Data":"1d6975855c3d80b832fda18e24ba2ace690d864d1ea2b451ae3f3913b8bfc2e3"} Jul 07 11:41:38.010206 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:38.010163 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" podStartSLOduration=1.806198562 podStartE2EDuration="2.01014586s" totalImagesPullingTime="203.947298ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:38.008674155 +0000 UTC m=+872.613010997" watchObservedRunningTime="2026-07-07 11:41:38.01014586 +0000 UTC m=+872.614482701" Jul 07 11:41:38.421452 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:38.421419 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:41:45.731361 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:45.731329 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:41:46.728782 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:46.728746 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:46.728976 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:46.728815 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:46.741280 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:46.741254 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:41:47.036054 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:41:47.035973 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6mkmq" Jul 07 11:42:06.017510 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:42:06.017402 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:42:06.018747 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:42:06.018686 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:43:21.027137 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:43:21.027102 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:43:30.426224 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:43:30.426144 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:43:42.421751 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:43:42.421712 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:43:50.326041 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:43:50.326010 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:44:01.520377 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:44:01.520342 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:44:27.036873 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:44:27.036838 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:44:39.822317 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:44:39.822267 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:45:14.826393 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:45:14.826358 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:45:24.518776 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:45:24.518741 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:45:33.319758 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:45:33.319720 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:45:42.521298 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:45:42.521261 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:45:53.325378 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:45:53.325347 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:46:04.121207 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:46:04.121174 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:46:15.620291 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:46:15.620251 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:46:24.717631 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:46:24.717596 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:46:47.919605 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:46:47.919507 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:46:59.123327 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:46:59.123286 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:47:06.040718 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:47:06.040612 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:47:06.042083 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:47:06.042064 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:47:22.727489 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:47:22.727451 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:47:33.716305 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:47:33.716271 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:47:48.717882 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:47:48.717845 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:47:59.125730 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:47:59.125695 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:48:08.823926 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:48:08.823893 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:48:18.017237 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:48:18.017200 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:48:27.817366 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:48:27.817333 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:50:17.133717 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:50:17.133678 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:50:25.927830 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:50:25.927796 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:50:39.628616 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:50:39.628579 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:50:49.322029 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:50:49.321995 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:51:06.421214 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:51:06.421174 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:51:16.521404 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:51:16.521367 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:51:28.020337 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:51:28.020300 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:51:36.618466 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:51:36.618430 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:51:47.019316 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:51:47.019286 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:52:01.222008 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:52:01.221973 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:52:06.064035 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:52:06.063923 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:52:06.068126 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:52:06.065520 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:52:12.024049 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:52:12.024014 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:52:27.421002 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:52:27.420915 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:52:34.620419 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:52:34.620378 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:52:43.826958 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:52:43.826916 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:52:53.318090 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:52:53.318056 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:53:10.518790 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:53:10.518753 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:53:19.720139 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:53:19.720101 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:53:29.127945 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:53:29.127905 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:53:40.019802 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:53:40.019765 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:53:49.238369 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:53:49.238333 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:53:58.121805 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:53:58.121718 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:54:11.626684 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:54:11.626648 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:54:20.819431 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:54:20.819389 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:54:30.232553 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:54:30.232506 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:54:42.512924 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:54:42.512890 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:54:53.120634 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:54:53.120597 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:55:01.937847 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:55:01.937817 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:55:13.628466 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:55:13.628430 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:55:25.428250 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:55:25.428210 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:55:34.644043 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:55:34.643964 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:55:47.627153 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:55:47.627117 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:55:59.176168 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:55:59.176137 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:56:10.815625 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:56:10.815584 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:56:18.879377 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:56:18.879341 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:56:40.922697 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:56:40.922659 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:56:49.542423 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:56:49.542386 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:57:06.087290 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:57:06.087181 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:57:06.093337 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:57:06.089645 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 11:58:13.324210 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:58:13.324175 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:58:21.821495 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:58:21.821461 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:58:32.931575 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:58:32.931485 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:58:43.621961 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:58:43.621927 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:59:15.420992 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:59:15.420958 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:59:30.616627 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:59:30.616581 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:59:44.115715 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:59:44.115685 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 11:59:53.720903 ip-10-0-139-127 kubenswrapper[2578]: I0707 11:59:53.720870 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:00:06.420137 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:00:06.420101 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:00:40.819776 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:00:40.819739 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:01:44.414607 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.414576 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7"] Jul 07 12:01:44.417602 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.417580 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:44.420327 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.420305 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-vld86\"" Jul 07 12:01:44.420474 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.420304 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-302b88-serving-cert\"" Jul 07 12:01:44.430347 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.430320 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7"] Jul 07 12:01:44.452645 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.452621 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/865f6c69-cc25-42ae-92bb-156649d9db1e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:44.452763 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.452655 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/865f6c69-cc25-42ae-92bb-156649d9db1e-maas-api-tls\") pod \"maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:44.452763 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.452704 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjnql\" (UniqueName: \"kubernetes.io/projected/865f6c69-cc25-42ae-92bb-156649d9db1e-kube-api-access-cjnql\") pod \"maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:44.554116 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.554071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjnql\" (UniqueName: \"kubernetes.io/projected/865f6c69-cc25-42ae-92bb-156649d9db1e-kube-api-access-cjnql\") pod \"maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:44.554288 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.554206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/865f6c69-cc25-42ae-92bb-156649d9db1e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:44.554288 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.554251 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/865f6c69-cc25-42ae-92bb-156649d9db1e-maas-api-tls\") pod \"maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:44.554373 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:01:44.554359 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-302b88-serving-cert: secret "maas-api-e2e-inf-a-302b88-serving-cert" not found Jul 07 12:01:44.554442 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:01:44.554432 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/865f6c69-cc25-42ae-92bb-156649d9db1e-maas-api-tls podName:865f6c69-cc25-42ae-92bb-156649d9db1e nodeName:}" failed. No retries permitted until 2026-07-07 12:01:45.054410241 +0000 UTC m=+2079.658747059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/865f6c69-cc25-42ae-92bb-156649d9db1e-maas-api-tls") pod "maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" (UID: "865f6c69-cc25-42ae-92bb-156649d9db1e") : secret "maas-api-e2e-inf-a-302b88-serving-cert" not found Jul 07 12:01:44.554879 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.554860 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/865f6c69-cc25-42ae-92bb-156649d9db1e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:44.563284 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:44.563264 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjnql\" (UniqueName: \"kubernetes.io/projected/865f6c69-cc25-42ae-92bb-156649d9db1e-kube-api-access-cjnql\") pod \"maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:45.058045 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:45.058004 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/865f6c69-cc25-42ae-92bb-156649d9db1e-maas-api-tls\") pod \"maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:45.060515 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:45.060484 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/865f6c69-cc25-42ae-92bb-156649d9db1e-maas-api-tls\") pod \"maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:45.329305 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:45.329206 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:45.465287 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:45.465263 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7"] Jul 07 12:01:45.467654 ip-10-0-139-127 kubenswrapper[2578]: W0707 12:01:45.467625 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod865f6c69_cc25_42ae_92bb_156649d9db1e.slice/crio-7942ec4c49aa60b5737e04096fff6b240e77aaeda158344ba58790a3990804e9 WatchSource:0}: Error finding container 7942ec4c49aa60b5737e04096fff6b240e77aaeda158344ba58790a3990804e9: Status 404 returned error can't find the container with id 7942ec4c49aa60b5737e04096fff6b240e77aaeda158344ba58790a3990804e9 Jul 07 12:01:45.468965 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:45.468950 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 12:01:45.906016 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:45.905983 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" event={"ID":"865f6c69-cc25-42ae-92bb-156649d9db1e","Type":"ContainerStarted","Data":"7942ec4c49aa60b5737e04096fff6b240e77aaeda158344ba58790a3990804e9"} Jul 07 12:01:47.923101 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:47.923067 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" event={"ID":"865f6c69-cc25-42ae-92bb-156649d9db1e","Type":"ContainerStarted","Data":"792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3"} Jul 07 12:01:47.923482 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:47.923154 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:01:47.940195 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:47.940139 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" podStartSLOduration=1.557424459 podStartE2EDuration="3.940122865s" totalImagesPullingTime="2.382698406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:01:47.939096957 +0000 UTC m=+2082.543433802" watchObservedRunningTime="2026-07-07 12:01:47.940122865 +0000 UTC m=+2082.544459715" Jul 07 12:01:48.019167 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:48.019131 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:01:53.937649 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:01:53.937617 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:02:01.715906 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:01.715872 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:02:06.115255 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:06.115154 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 12:02:06.118563 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:06.117815 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 12:02:11.021712 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:11.021678 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:02:27.017762 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:27.017722 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:02:41.116665 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:41.116621 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7"] Jul 07 12:02:41.117262 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:41.116871 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" podUID="865f6c69-cc25-42ae-92bb-156649d9db1e" containerName="maas-api" containerID="cri-o://792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3" gracePeriod=30 Jul 07 12:02:41.568244 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:41.568220 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:02:41.646310 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:41.646282 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/865f6c69-cc25-42ae-92bb-156649d9db1e-openshift-service-ca\") pod \"865f6c69-cc25-42ae-92bb-156649d9db1e\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " Jul 07 12:02:41.646462 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:41.646339 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/865f6c69-cc25-42ae-92bb-156649d9db1e-maas-api-tls\") pod \"865f6c69-cc25-42ae-92bb-156649d9db1e\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " Jul 07 12:02:41.646462 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:41.646375 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjnql\" (UniqueName: \"kubernetes.io/projected/865f6c69-cc25-42ae-92bb-156649d9db1e-kube-api-access-cjnql\") pod \"865f6c69-cc25-42ae-92bb-156649d9db1e\" (UID: \"865f6c69-cc25-42ae-92bb-156649d9db1e\") " Jul 07 12:02:41.646656 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:41.646634 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/865f6c69-cc25-42ae-92bb-156649d9db1e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "865f6c69-cc25-42ae-92bb-156649d9db1e" (UID: "865f6c69-cc25-42ae-92bb-156649d9db1e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:02:41.648501 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:41.648451 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865f6c69-cc25-42ae-92bb-156649d9db1e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "865f6c69-cc25-42ae-92bb-156649d9db1e" (UID: "865f6c69-cc25-42ae-92bb-156649d9db1e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:02:41.648602 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:41.648498 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/865f6c69-cc25-42ae-92bb-156649d9db1e-kube-api-access-cjnql" (OuterVolumeSpecName: "kube-api-access-cjnql") pod "865f6c69-cc25-42ae-92bb-156649d9db1e" (UID: "865f6c69-cc25-42ae-92bb-156649d9db1e"). InnerVolumeSpecName "kube-api-access-cjnql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:02:41.747257 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:41.747228 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjnql\" (UniqueName: \"kubernetes.io/projected/865f6c69-cc25-42ae-92bb-156649d9db1e-kube-api-access-cjnql\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:02:41.747257 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:41.747254 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/865f6c69-cc25-42ae-92bb-156649d9db1e-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:02:41.747257 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:41.747264 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/865f6c69-cc25-42ae-92bb-156649d9db1e-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:02:42.143043 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:42.143012 2578 generic.go:356] "Generic (PLEG): container finished" podID="865f6c69-cc25-42ae-92bb-156649d9db1e" containerID="792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3" exitCode=0 Jul 07 12:02:42.143434 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:42.143056 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" Jul 07 12:02:42.143434 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:42.143090 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" event={"ID":"865f6c69-cc25-42ae-92bb-156649d9db1e","Type":"ContainerDied","Data":"792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3"} Jul 07 12:02:42.143434 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:42.143144 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7" event={"ID":"865f6c69-cc25-42ae-92bb-156649d9db1e","Type":"ContainerDied","Data":"7942ec4c49aa60b5737e04096fff6b240e77aaeda158344ba58790a3990804e9"} Jul 07 12:02:42.143434 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:42.143167 2578 scope.go:117] "RemoveContainer" containerID="792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3" Jul 07 12:02:42.153455 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:42.153421 2578 scope.go:117] "RemoveContainer" containerID="792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3" Jul 07 12:02:42.153804 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:02:42.153773 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3\": container with ID starting with 792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3 not found: ID does not exist" containerID="792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3" Jul 07 12:02:42.153881 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:42.153814 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3"} err="failed to get container status \"792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3\": rpc error: code = NotFound desc = could not find container \"792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3\": container with ID starting with 792ebb2215f5f1c773feff1147ef88f0e5e2f224b5932fe50aee8f537ddfaba3 not found: ID does not exist" Jul 07 12:02:42.159463 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:42.159433 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7"] Jul 07 12:02:42.165428 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:42.165396 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-302b88-c6868d4b4-vxhm7"] Jul 07 12:02:43.940418 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:43.940389 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="865f6c69-cc25-42ae-92bb-156649d9db1e" path="/var/lib/kubelet/pods/865f6c69-cc25-42ae-92bb-156649d9db1e/volumes" Jul 07 12:02:44.217725 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:02:44.217642 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:03:26.819487 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:26.819454 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:03:33.938597 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:33.938567 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p"] Jul 07 12:03:33.939058 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:33.938853 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="865f6c69-cc25-42ae-92bb-156649d9db1e" containerName="maas-api" Jul 07 12:03:33.939058 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:33.938863 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="865f6c69-cc25-42ae-92bb-156649d9db1e" containerName="maas-api" Jul 07 12:03:33.939058 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:33.938934 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="865f6c69-cc25-42ae-92bb-156649d9db1e" containerName="maas-api" Jul 07 12:03:33.941874 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:33.941852 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:33.944372 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:33.944352 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-vld86\"" Jul 07 12:03:33.944469 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:33.944454 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-ecfb77-serving-cert\"" Jul 07 12:03:33.948446 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:33.948419 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p"] Jul 07 12:03:34.003608 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:34.003573 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-maas-api-tls\") pod \"maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p\" (UID: \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\") " pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:34.003811 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:34.003631 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-openshift-service-ca\") pod \"maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p\" (UID: \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\") " pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:34.003811 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:34.003689 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp4wx\" (UniqueName: \"kubernetes.io/projected/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-kube-api-access-jp4wx\") pod \"maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p\" (UID: \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\") " pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:34.104814 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:34.104778 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-openshift-service-ca\") pod \"maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p\" (UID: \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\") " pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:34.104988 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:34.104828 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp4wx\" (UniqueName: \"kubernetes.io/projected/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-kube-api-access-jp4wx\") pod \"maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p\" (UID: \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\") " pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:34.104988 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:34.104886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-maas-api-tls\") pod \"maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p\" (UID: \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\") " pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:34.105524 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:34.105501 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-openshift-service-ca\") pod \"maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p\" (UID: \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\") " pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:34.107329 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:34.107306 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-maas-api-tls\") pod \"maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p\" (UID: \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\") " pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:34.113574 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:34.113513 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp4wx\" (UniqueName: \"kubernetes.io/projected/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-kube-api-access-jp4wx\") pod \"maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p\" (UID: \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\") " pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:34.253756 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:34.253664 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:34.429137 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:34.429116 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p"] Jul 07 12:03:34.431162 ip-10-0-139-127 kubenswrapper[2578]: W0707 12:03:34.431133 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b0ab36b_a0b6_4f0e_97f2_9495571696c3.slice/crio-c4118e57308579b72580a7a7212e685953c641f5c30b95a7037f1492f4f96071 WatchSource:0}: Error finding container c4118e57308579b72580a7a7212e685953c641f5c30b95a7037f1492f4f96071: Status 404 returned error can't find the container with id c4118e57308579b72580a7a7212e685953c641f5c30b95a7037f1492f4f96071 Jul 07 12:03:35.370960 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:35.370927 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" event={"ID":"7b0ab36b-a0b6-4f0e-97f2-9495571696c3","Type":"ContainerStarted","Data":"2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b"} Jul 07 12:03:35.370960 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:35.370977 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" event={"ID":"7b0ab36b-a0b6-4f0e-97f2-9495571696c3","Type":"ContainerStarted","Data":"c4118e57308579b72580a7a7212e685953c641f5c30b95a7037f1492f4f96071"} Jul 07 12:03:35.371467 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:35.371022 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:35.390532 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:35.390484 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" podStartSLOduration=2.09102284 podStartE2EDuration="2.390470064s" totalImagesPullingTime="299.447224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:03:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:03:35.388754565 +0000 UTC m=+2189.993091413" watchObservedRunningTime="2026-07-07 12:03:35.390470064 +0000 UTC m=+2189.994806904" Jul 07 12:03:41.379656 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:41.379629 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:41.820428 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:41.820386 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:03:42.633667 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:42.633630 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4"] Jul 07 12:03:42.636940 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:42.636917 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:42.639051 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:42.639032 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-a1dd3f-serving-cert\"" Jul 07 12:03:42.652395 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:42.648346 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4"] Jul 07 12:03:42.778690 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:42.778658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c1ca03-aea3-4cba-83b4-40485713a0d3-openshift-service-ca\") pod \"maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:42.778860 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:42.778744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c1ca03-aea3-4cba-83b4-40485713a0d3-maas-api-tls\") pod \"maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:42.778860 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:42.778762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s77lm\" (UniqueName: \"kubernetes.io/projected/43c1ca03-aea3-4cba-83b4-40485713a0d3-kube-api-access-s77lm\") pod \"maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:42.879343 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:42.879309 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c1ca03-aea3-4cba-83b4-40485713a0d3-maas-api-tls\") pod \"maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:42.879343 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:42.879345 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77lm\" (UniqueName: \"kubernetes.io/projected/43c1ca03-aea3-4cba-83b4-40485713a0d3-kube-api-access-s77lm\") pod \"maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:42.879526 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:42.879387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c1ca03-aea3-4cba-83b4-40485713a0d3-openshift-service-ca\") pod \"maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:42.879526 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:03:42.879473 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-a1dd3f-serving-cert: secret "maas-api-e2e-api-b-a1dd3f-serving-cert" not found Jul 07 12:03:42.879622 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:03:42.879563 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c1ca03-aea3-4cba-83b4-40485713a0d3-maas-api-tls podName:43c1ca03-aea3-4cba-83b4-40485713a0d3 nodeName:}" failed. No retries permitted until 2026-07-07 12:03:43.379520948 +0000 UTC m=+2197.983857768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/43c1ca03-aea3-4cba-83b4-40485713a0d3-maas-api-tls") pod "maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" (UID: "43c1ca03-aea3-4cba-83b4-40485713a0d3") : secret "maas-api-e2e-api-b-a1dd3f-serving-cert" not found Jul 07 12:03:42.880037 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:42.880020 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c1ca03-aea3-4cba-83b4-40485713a0d3-openshift-service-ca\") pod \"maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:42.904390 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:42.904323 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s77lm\" (UniqueName: \"kubernetes.io/projected/43c1ca03-aea3-4cba-83b4-40485713a0d3-kube-api-access-s77lm\") pod \"maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:43.385242 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:43.385203 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c1ca03-aea3-4cba-83b4-40485713a0d3-maas-api-tls\") pod \"maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:43.387622 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:43.387596 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c1ca03-aea3-4cba-83b4-40485713a0d3-maas-api-tls\") pod \"maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:43.557756 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:43.557716 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:43.686123 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:43.686059 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4"] Jul 07 12:03:43.688523 ip-10-0-139-127 kubenswrapper[2578]: W0707 12:03:43.688497 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c1ca03_aea3_4cba_83b4_40485713a0d3.slice/crio-910f67c7c554f2c502fd7c7bf7a3c5142f6f184ebcd07121764e97bd6cd2a23f WatchSource:0}: Error finding container 910f67c7c554f2c502fd7c7bf7a3c5142f6f184ebcd07121764e97bd6cd2a23f: Status 404 returned error can't find the container with id 910f67c7c554f2c502fd7c7bf7a3c5142f6f184ebcd07121764e97bd6cd2a23f Jul 07 12:03:44.409891 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:44.409799 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" event={"ID":"43c1ca03-aea3-4cba-83b4-40485713a0d3","Type":"ContainerStarted","Data":"c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1"} Jul 07 12:03:44.409891 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:44.409856 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" event={"ID":"43c1ca03-aea3-4cba-83b4-40485713a0d3","Type":"ContainerStarted","Data":"910f67c7c554f2c502fd7c7bf7a3c5142f6f184ebcd07121764e97bd6cd2a23f"} Jul 07 12:03:44.409891 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:44.409885 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:44.429009 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:44.428961 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" podStartSLOduration=1.995912964 podStartE2EDuration="2.428946833s" totalImagesPullingTime="433.033869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:03:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:03:44.427091326 +0000 UTC m=+2199.031428188" watchObservedRunningTime="2026-07-07 12:03:44.428946833 +0000 UTC m=+2199.033283673" Jul 07 12:03:50.418964 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:50.418937 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:03:55.612611 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:55.612575 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p"] Jul 07 12:03:55.613127 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:55.612792 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" podUID="7b0ab36b-a0b6-4f0e-97f2-9495571696c3" containerName="maas-api" containerID="cri-o://2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b" gracePeriod=30 Jul 07 12:03:55.872648 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:55.872582 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:55.993878 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:55.993830 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jp4wx\" (UniqueName: \"kubernetes.io/projected/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-kube-api-access-jp4wx\") pod \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\" (UID: \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\") " Jul 07 12:03:55.994056 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:55.993927 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-maas-api-tls\") pod \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\" (UID: \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\") " Jul 07 12:03:55.994056 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:55.993960 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-openshift-service-ca\") pod \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\" (UID: \"7b0ab36b-a0b6-4f0e-97f2-9495571696c3\") " Jul 07 12:03:55.994534 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:55.994504 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7b0ab36b-a0b6-4f0e-97f2-9495571696c3" (UID: "7b0ab36b-a0b6-4f0e-97f2-9495571696c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:03:55.996157 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:55.996133 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-kube-api-access-jp4wx" (OuterVolumeSpecName: "kube-api-access-jp4wx") pod "7b0ab36b-a0b6-4f0e-97f2-9495571696c3" (UID: "7b0ab36b-a0b6-4f0e-97f2-9495571696c3"). InnerVolumeSpecName "kube-api-access-jp4wx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:03:55.996157 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:55.996148 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7b0ab36b-a0b6-4f0e-97f2-9495571696c3" (UID: "7b0ab36b-a0b6-4f0e-97f2-9495571696c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:03:56.095649 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:56.095614 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp4wx\" (UniqueName: \"kubernetes.io/projected/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-kube-api-access-jp4wx\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:03:56.095649 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:56.095643 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:03:56.095649 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:56.095653 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b0ab36b-a0b6-4f0e-97f2-9495571696c3-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:03:56.460345 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:56.460309 2578 generic.go:356] "Generic (PLEG): container finished" podID="7b0ab36b-a0b6-4f0e-97f2-9495571696c3" containerID="2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b" exitCode=0 Jul 07 12:03:56.460526 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:56.460372 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" Jul 07 12:03:56.460526 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:56.460399 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" event={"ID":"7b0ab36b-a0b6-4f0e-97f2-9495571696c3","Type":"ContainerDied","Data":"2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b"} Jul 07 12:03:56.460526 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:56.460456 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p" event={"ID":"7b0ab36b-a0b6-4f0e-97f2-9495571696c3","Type":"ContainerDied","Data":"c4118e57308579b72580a7a7212e685953c641f5c30b95a7037f1492f4f96071"} Jul 07 12:03:56.460526 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:56.460478 2578 scope.go:117] "RemoveContainer" containerID="2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b" Jul 07 12:03:56.469085 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:56.469062 2578 scope.go:117] "RemoveContainer" containerID="2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b" Jul 07 12:03:56.469327 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:03:56.469308 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b\": container with ID starting with 2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b not found: ID does not exist" containerID="2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b" Jul 07 12:03:56.469394 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:56.469336 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b"} err="failed to get container status \"2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b\": rpc error: code = NotFound desc = could not find container \"2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b\": container with ID starting with 2a123b521bb6bc1ca32506b3f2624d23ccbcd09b744a93b379fb27f6d4f0c33b not found: ID does not exist" Jul 07 12:03:56.480934 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:56.480906 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p"] Jul 07 12:03:56.486490 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:56.486463 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-ecfb77-6cd478db7b-cmk9p"] Jul 07 12:03:57.938013 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:57.937979 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b0ab36b-a0b6-4f0e-97f2-9495571696c3" path="/var/lib/kubelet/pods/7b0ab36b-a0b6-4f0e-97f2-9495571696c3/volumes" Jul 07 12:03:58.913992 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:03:58.913965 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:04:02.661017 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:02.660972 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4"] Jul 07 12:04:02.661465 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:02.661218 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" podUID="43c1ca03-aea3-4cba-83b4-40485713a0d3" containerName="maas-api" containerID="cri-o://c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1" gracePeriod=30 Jul 07 12:04:02.906727 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:02.906697 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:04:03.055482 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.055454 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s77lm\" (UniqueName: \"kubernetes.io/projected/43c1ca03-aea3-4cba-83b4-40485713a0d3-kube-api-access-s77lm\") pod \"43c1ca03-aea3-4cba-83b4-40485713a0d3\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " Jul 07 12:04:03.055685 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.055503 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c1ca03-aea3-4cba-83b4-40485713a0d3-maas-api-tls\") pod \"43c1ca03-aea3-4cba-83b4-40485713a0d3\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " Jul 07 12:04:03.055756 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.055712 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c1ca03-aea3-4cba-83b4-40485713a0d3-openshift-service-ca\") pod \"43c1ca03-aea3-4cba-83b4-40485713a0d3\" (UID: \"43c1ca03-aea3-4cba-83b4-40485713a0d3\") " Jul 07 12:04:03.056030 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.056004 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c1ca03-aea3-4cba-83b4-40485713a0d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "43c1ca03-aea3-4cba-83b4-40485713a0d3" (UID: "43c1ca03-aea3-4cba-83b4-40485713a0d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:04:03.057587 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.057562 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43c1ca03-aea3-4cba-83b4-40485713a0d3-kube-api-access-s77lm" (OuterVolumeSpecName: "kube-api-access-s77lm") pod "43c1ca03-aea3-4cba-83b4-40485713a0d3" (UID: "43c1ca03-aea3-4cba-83b4-40485713a0d3"). InnerVolumeSpecName "kube-api-access-s77lm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:04:03.057665 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.057603 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c1ca03-aea3-4cba-83b4-40485713a0d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "43c1ca03-aea3-4cba-83b4-40485713a0d3" (UID: "43c1ca03-aea3-4cba-83b4-40485713a0d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:04:03.157315 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.157281 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c1ca03-aea3-4cba-83b4-40485713a0d3-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:04:03.157315 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.157309 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s77lm\" (UniqueName: \"kubernetes.io/projected/43c1ca03-aea3-4cba-83b4-40485713a0d3-kube-api-access-s77lm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:04:03.157315 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.157318 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c1ca03-aea3-4cba-83b4-40485713a0d3-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:04:03.492168 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.492129 2578 generic.go:356] "Generic (PLEG): container finished" podID="43c1ca03-aea3-4cba-83b4-40485713a0d3" containerID="c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1" exitCode=0 Jul 07 12:04:03.492340 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.492175 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" Jul 07 12:04:03.492340 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.492202 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" event={"ID":"43c1ca03-aea3-4cba-83b4-40485713a0d3","Type":"ContainerDied","Data":"c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1"} Jul 07 12:04:03.492340 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.492253 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4" event={"ID":"43c1ca03-aea3-4cba-83b4-40485713a0d3","Type":"ContainerDied","Data":"910f67c7c554f2c502fd7c7bf7a3c5142f6f184ebcd07121764e97bd6cd2a23f"} Jul 07 12:04:03.492340 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.492278 2578 scope.go:117] "RemoveContainer" containerID="c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1" Jul 07 12:04:03.500962 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.500947 2578 scope.go:117] "RemoveContainer" containerID="c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1" Jul 07 12:04:03.501218 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:04:03.501197 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1\": container with ID starting with c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1 not found: ID does not exist" containerID="c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1" Jul 07 12:04:03.501292 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.501227 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1"} err="failed to get container status \"c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1\": rpc error: code = NotFound desc = could not find container \"c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1\": container with ID starting with c5093ef4315954c7acb182cd7efeca4fe6bab616a150d421039caea0e63b32c1 not found: ID does not exist" Jul 07 12:04:03.512242 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.512220 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4"] Jul 07 12:04:03.518861 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.518842 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-a1dd3f-8b467647d-7lqf4"] Jul 07 12:04:03.937453 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:03.937425 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43c1ca03-aea3-4cba-83b4-40485713a0d3" path="/var/lib/kubelet/pods/43c1ca03-aea3-4cba-83b4-40485713a0d3/volumes" Jul 07 12:04:10.715626 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:10.715591 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:04:12.116717 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.116685 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9"] Jul 07 12:04:12.117075 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.116969 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b0ab36b-a0b6-4f0e-97f2-9495571696c3" containerName="maas-api" Jul 07 12:04:12.117075 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.116979 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b0ab36b-a0b6-4f0e-97f2-9495571696c3" containerName="maas-api" Jul 07 12:04:12.117075 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.117001 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c1ca03-aea3-4cba-83b4-40485713a0d3" containerName="maas-api" Jul 07 12:04:12.117075 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.117007 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c1ca03-aea3-4cba-83b4-40485713a0d3" containerName="maas-api" Jul 07 12:04:12.117075 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.117050 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b0ab36b-a0b6-4f0e-97f2-9495571696c3" containerName="maas-api" Jul 07 12:04:12.117075 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.117059 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c1ca03-aea3-4cba-83b4-40485713a0d3" containerName="maas-api" Jul 07 12:04:12.119648 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.119623 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:12.122744 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.122725 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-vld86\"" Jul 07 12:04:12.122986 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.122972 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-10f898-serving-cert\"" Jul 07 12:04:12.125571 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.125530 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzggb\" (UniqueName: \"kubernetes.io/projected/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-kube-api-access-vzggb\") pod \"maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:12.125652 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.125616 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-openshift-service-ca\") pod \"maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:12.125700 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.125678 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-maas-api-tls\") pod \"maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:12.139622 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.139601 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9"] Jul 07 12:04:12.226061 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.226031 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-openshift-service-ca\") pod \"maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:12.226233 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.226077 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-maas-api-tls\") pod \"maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:12.226233 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.226133 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzggb\" (UniqueName: \"kubernetes.io/projected/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-kube-api-access-vzggb\") pod \"maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:12.226233 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:04:12.226185 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-10f898-serving-cert: secret "maas-api-e2e-shared-a-10f898-serving-cert" not found Jul 07 12:04:12.226364 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:04:12.226266 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-maas-api-tls podName:e39ee4fb-c2f2-48fb-99a9-9728ab3e9358 nodeName:}" failed. No retries permitted until 2026-07-07 12:04:12.726235638 +0000 UTC m=+2227.330572474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-maas-api-tls") pod "maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" (UID: "e39ee4fb-c2f2-48fb-99a9-9728ab3e9358") : secret "maas-api-e2e-shared-a-10f898-serving-cert" not found Jul 07 12:04:12.226879 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.226851 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-openshift-service-ca\") pod \"maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:12.235551 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.235519 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzggb\" (UniqueName: \"kubernetes.io/projected/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-kube-api-access-vzggb\") pod \"maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:12.728999 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:12.728959 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-maas-api-tls\") pod \"maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:12.729186 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:04:12.729098 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-10f898-serving-cert: secret "maas-api-e2e-shared-a-10f898-serving-cert" not found Jul 07 12:04:12.729186 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:04:12.729172 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-maas-api-tls podName:e39ee4fb-c2f2-48fb-99a9-9728ab3e9358 nodeName:}" failed. No retries permitted until 2026-07-07 12:04:13.729156452 +0000 UTC m=+2228.333493271 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-maas-api-tls") pod "maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" (UID: "e39ee4fb-c2f2-48fb-99a9-9728ab3e9358") : secret "maas-api-e2e-shared-a-10f898-serving-cert" not found Jul 07 12:04:13.736816 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:13.736784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-maas-api-tls\") pod \"maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:13.739200 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:13.739179 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-maas-api-tls\") pod \"maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:13.932875 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:13.932836 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:14.064733 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:14.064701 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9"] Jul 07 12:04:14.066685 ip-10-0-139-127 kubenswrapper[2578]: W0707 12:04:14.066660 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39ee4fb_c2f2_48fb_99a9_9728ab3e9358.slice/crio-287a9bdc67e5fcde9911f473cd5a1865882a93903fde00cc2746c69136700a27 WatchSource:0}: Error finding container 287a9bdc67e5fcde9911f473cd5a1865882a93903fde00cc2746c69136700a27: Status 404 returned error can't find the container with id 287a9bdc67e5fcde9911f473cd5a1865882a93903fde00cc2746c69136700a27 Jul 07 12:04:14.540556 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:14.540493 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" event={"ID":"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358","Type":"ContainerStarted","Data":"287a9bdc67e5fcde9911f473cd5a1865882a93903fde00cc2746c69136700a27"} Jul 07 12:04:15.546457 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:15.546414 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" event={"ID":"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358","Type":"ContainerStarted","Data":"76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3"} Jul 07 12:04:15.546857 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:15.546576 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:15.565286 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:15.565239 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" podStartSLOduration=3.043530879 podStartE2EDuration="3.565225651s" totalImagesPullingTime="521.694772ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:04:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:04:15.56408 +0000 UTC m=+2230.168416843" watchObservedRunningTime="2026-07-07 12:04:15.565225651 +0000 UTC m=+2230.169562470" Jul 07 12:04:19.765517 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:19.765482 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl"] Jul 07 12:04:19.768966 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:19.768950 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:19.771449 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:19.771424 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-068f74-serving-cert\"" Jul 07 12:04:19.779390 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:19.779367 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl"] Jul 07 12:04:19.786514 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:19.786488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnkjc\" (UniqueName: \"kubernetes.io/projected/f92f4576-0839-45e2-9b97-4f4e481828c0-kube-api-access-mnkjc\") pod \"maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:19.786618 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:19.786583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f92f4576-0839-45e2-9b97-4f4e481828c0-maas-api-tls\") pod \"maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:19.786654 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:19.786615 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92f4576-0839-45e2-9b97-4f4e481828c0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:19.887061 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:19.887028 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnkjc\" (UniqueName: \"kubernetes.io/projected/f92f4576-0839-45e2-9b97-4f4e481828c0-kube-api-access-mnkjc\") pod \"maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:19.887228 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:19.887072 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f92f4576-0839-45e2-9b97-4f4e481828c0-maas-api-tls\") pod \"maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:19.887228 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:19.887090 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92f4576-0839-45e2-9b97-4f4e481828c0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:19.887228 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:04:19.887181 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-068f74-serving-cert: secret "maas-api-e2e-shared-b-068f74-serving-cert" not found Jul 07 12:04:19.887363 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:04:19.887243 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f92f4576-0839-45e2-9b97-4f4e481828c0-maas-api-tls podName:f92f4576-0839-45e2-9b97-4f4e481828c0 nodeName:}" failed. No retries permitted until 2026-07-07 12:04:20.38722436 +0000 UTC m=+2234.991561182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f92f4576-0839-45e2-9b97-4f4e481828c0-maas-api-tls") pod "maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" (UID: "f92f4576-0839-45e2-9b97-4f4e481828c0") : secret "maas-api-e2e-shared-b-068f74-serving-cert" not found Jul 07 12:04:19.887675 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:19.887659 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92f4576-0839-45e2-9b97-4f4e481828c0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:19.895181 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:19.895151 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnkjc\" (UniqueName: \"kubernetes.io/projected/f92f4576-0839-45e2-9b97-4f4e481828c0-kube-api-access-mnkjc\") pod \"maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:20.126048 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:20.125972 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:04:20.392118 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:20.392036 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f92f4576-0839-45e2-9b97-4f4e481828c0-maas-api-tls\") pod \"maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:20.394360 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:20.394331 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f92f4576-0839-45e2-9b97-4f4e481828c0-maas-api-tls\") pod \"maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:20.680337 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:20.680304 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:20.809634 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:20.809488 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl"] Jul 07 12:04:21.556416 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:21.556340 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:04:21.573235 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:21.573114 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" event={"ID":"f92f4576-0839-45e2-9b97-4f4e481828c0","Type":"ContainerStarted","Data":"49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486"} Jul 07 12:04:21.573235 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:21.573195 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" event={"ID":"f92f4576-0839-45e2-9b97-4f4e481828c0","Type":"ContainerStarted","Data":"2ba61f400e32f82ca3eb935185a0e85cf2387268d19cdbb4e99bc87c5fbbafb3"} Jul 07 12:04:21.573235 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:21.573236 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:21.589401 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:21.589351 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" podStartSLOduration=2.199666945 podStartE2EDuration="2.589336479s" totalImagesPullingTime="389.669534ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:04:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:04:21.588058648 +0000 UTC m=+2236.192395492" watchObservedRunningTime="2026-07-07 12:04:21.589336479 +0000 UTC m=+2236.193673319" Jul 07 12:04:27.582975 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:27.582898 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:04:31.511398 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:31.511358 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:04:47.916703 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:04:47.916662 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:05:01.219483 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:05:01.219445 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:05:14.925293 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:05:14.925251 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:05:32.827908 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:05:32.827871 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:05:50.617921 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:05:50.617883 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:06:07.722499 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:06:07.722450 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:06:24.014186 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:06:24.014155 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:06:41.026479 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:06:41.026444 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:06:56.413587 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:06:56.413551 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:07:06.141678 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:07:06.141550 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 12:07:06.146965 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:07:06.144831 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r54t_5ae6d83b-0067-4e51-b9a8-26b4c2917b78/ovn-acl-logging/0.log" Jul 07 12:07:11.317126 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:07:11.317090 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:07:30.621867 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:07:30.621782 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:07:49.118514 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:07:49.118473 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:08:06.721339 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:08:06.721303 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:08:22.524992 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:08:22.524960 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:08:48.618997 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:08:48.618961 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:09:01.920442 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:01.920406 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:09:23.413586 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:23.413531 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:09:30.762876 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:30.762839 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9"] Jul 07 12:09:30.763353 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:30.763197 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" podUID="e39ee4fb-c2f2-48fb-99a9-9728ab3e9358" containerName="maas-api" containerID="cri-o://76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3" gracePeriod=30 Jul 07 12:09:31.200023 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.199998 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:09:31.304081 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.303977 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-openshift-service-ca\") pod \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " Jul 07 12:09:31.304081 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.304040 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-maas-api-tls\") pod \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " Jul 07 12:09:31.304081 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.304078 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzggb\" (UniqueName: \"kubernetes.io/projected/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-kube-api-access-vzggb\") pod \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\" (UID: \"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358\") " Jul 07 12:09:31.304462 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.304434 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e39ee4fb-c2f2-48fb-99a9-9728ab3e9358" (UID: "e39ee4fb-c2f2-48fb-99a9-9728ab3e9358"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:09:31.306165 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.306145 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e39ee4fb-c2f2-48fb-99a9-9728ab3e9358" (UID: "e39ee4fb-c2f2-48fb-99a9-9728ab3e9358"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:09:31.306263 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.306243 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-kube-api-access-vzggb" (OuterVolumeSpecName: "kube-api-access-vzggb") pod "e39ee4fb-c2f2-48fb-99a9-9728ab3e9358" (UID: "e39ee4fb-c2f2-48fb-99a9-9728ab3e9358"). InnerVolumeSpecName "kube-api-access-vzggb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:09:31.405374 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.405330 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzggb\" (UniqueName: \"kubernetes.io/projected/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-kube-api-access-vzggb\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:09:31.405374 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.405364 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:09:31.405374 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.405374 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:09:31.869756 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.869718 2578 generic.go:356] "Generic (PLEG): container finished" podID="e39ee4fb-c2f2-48fb-99a9-9728ab3e9358" containerID="76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3" exitCode=0 Jul 07 12:09:31.870206 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.869766 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" Jul 07 12:09:31.870206 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.869807 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" event={"ID":"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358","Type":"ContainerDied","Data":"76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3"} Jul 07 12:09:31.870206 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.869867 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9" event={"ID":"e39ee4fb-c2f2-48fb-99a9-9728ab3e9358","Type":"ContainerDied","Data":"287a9bdc67e5fcde9911f473cd5a1865882a93903fde00cc2746c69136700a27"} Jul 07 12:09:31.870206 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.869896 2578 scope.go:117] "RemoveContainer" containerID="76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3" Jul 07 12:09:31.878789 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.878762 2578 scope.go:117] "RemoveContainer" containerID="76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3" Jul 07 12:09:31.879117 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:09:31.879097 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3\": container with ID starting with 76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3 not found: ID does not exist" containerID="76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3" Jul 07 12:09:31.879166 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.879125 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3"} err="failed to get container status \"76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3\": rpc error: code = NotFound desc = could not find container \"76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3\": container with ID starting with 76038f6e94653562b14e565b153173c568598a504620644d9aa603d750434fa3 not found: ID does not exist" Jul 07 12:09:31.893431 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.893400 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9"] Jul 07 12:09:31.897334 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.897310 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-10f898-7fccb8bc8c-lnxl9"] Jul 07 12:09:31.938588 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:31.938557 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e39ee4fb-c2f2-48fb-99a9-9728ab3e9358" path="/var/lib/kubelet/pods/e39ee4fb-c2f2-48fb-99a9-9728ab3e9358/volumes" Jul 07 12:09:33.818502 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:09:33.818468 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:10:13.407698 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:13.407657 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl"] Jul 07 12:10:13.408226 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:13.407920 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" podUID="f92f4576-0839-45e2-9b97-4f4e481828c0" containerName="maas-api" containerID="cri-o://49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486" gracePeriod=30 Jul 07 12:10:13.854176 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:13.854153 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:10:13.976566 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:13.971026 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f92f4576-0839-45e2-9b97-4f4e481828c0-maas-api-tls\") pod \"f92f4576-0839-45e2-9b97-4f4e481828c0\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " Jul 07 12:10:13.976566 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:13.971129 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92f4576-0839-45e2-9b97-4f4e481828c0-openshift-service-ca\") pod \"f92f4576-0839-45e2-9b97-4f4e481828c0\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " Jul 07 12:10:13.976566 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:13.971174 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnkjc\" (UniqueName: \"kubernetes.io/projected/f92f4576-0839-45e2-9b97-4f4e481828c0-kube-api-access-mnkjc\") pod \"f92f4576-0839-45e2-9b97-4f4e481828c0\" (UID: \"f92f4576-0839-45e2-9b97-4f4e481828c0\") " Jul 07 12:10:13.976566 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:13.974238 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f92f4576-0839-45e2-9b97-4f4e481828c0-kube-api-access-mnkjc" (OuterVolumeSpecName: "kube-api-access-mnkjc") pod "f92f4576-0839-45e2-9b97-4f4e481828c0" (UID: "f92f4576-0839-45e2-9b97-4f4e481828c0"). InnerVolumeSpecName "kube-api-access-mnkjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:10:13.976566 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:13.976426 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f92f4576-0839-45e2-9b97-4f4e481828c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f92f4576-0839-45e2-9b97-4f4e481828c0" (UID: "f92f4576-0839-45e2-9b97-4f4e481828c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:10:13.976907 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:13.976832 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f92f4576-0839-45e2-9b97-4f4e481828c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f92f4576-0839-45e2-9b97-4f4e481828c0" (UID: "f92f4576-0839-45e2-9b97-4f4e481828c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:10:14.051728 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:14.051690 2578 generic.go:356] "Generic (PLEG): container finished" podID="f92f4576-0839-45e2-9b97-4f4e481828c0" containerID="49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486" exitCode=0 Jul 07 12:10:14.051900 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:14.051748 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" event={"ID":"f92f4576-0839-45e2-9b97-4f4e481828c0","Type":"ContainerDied","Data":"49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486"} Jul 07 12:10:14.051900 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:14.051761 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" Jul 07 12:10:14.051900 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:14.051832 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl" event={"ID":"f92f4576-0839-45e2-9b97-4f4e481828c0","Type":"ContainerDied","Data":"2ba61f400e32f82ca3eb935185a0e85cf2387268d19cdbb4e99bc87c5fbbafb3"} Jul 07 12:10:14.051900 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:14.051864 2578 scope.go:117] "RemoveContainer" containerID="49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486" Jul 07 12:10:14.063547 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:14.063503 2578 scope.go:117] "RemoveContainer" containerID="49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486" Jul 07 12:10:14.063903 ip-10-0-139-127 kubenswrapper[2578]: E0707 12:10:14.063885 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486\": container with ID starting with 49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486 not found: ID does not exist" containerID="49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486" Jul 07 12:10:14.063968 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:14.063914 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486"} err="failed to get container status \"49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486\": rpc error: code = NotFound desc = could not find container \"49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486\": container with ID starting with 49f1054ec69a7587375497334a322611542a7c4698e60d5618fa10f99fcf7486 not found: ID does not exist" Jul 07 12:10:14.071948 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:14.071921 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl"] Jul 07 12:10:14.072346 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:14.072307 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92f4576-0839-45e2-9b97-4f4e481828c0-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:10:14.072459 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:14.072336 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnkjc\" (UniqueName: \"kubernetes.io/projected/f92f4576-0839-45e2-9b97-4f4e481828c0-kube-api-access-mnkjc\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:10:14.072459 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:14.072377 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f92f4576-0839-45e2-9b97-4f4e481828c0-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\"" Jul 07 12:10:14.073802 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:14.073784 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-068f74-77dcb4d65c-54rdl"] Jul 07 12:10:15.522328 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:15.522284 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xtmx4"] Jul 07 12:10:15.937510 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:15.937479 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f92f4576-0839-45e2-9b97-4f4e481828c0" path="/var/lib/kubelet/pods/f92f4576-0839-45e2-9b97-4f4e481828c0/volumes" Jul 07 12:10:26.041118 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:26.041089 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f68d9489b-gfv5k_6c48f684-542f-438a-91a7-d924a8027930/copy-manifests/0.log" Jul 07 12:10:26.054752 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:26.054725 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f68d9489b-gfv5k_6c48f684-542f-438a-91a7-d924a8027930/manager/0.log" Jul 07 12:10:28.468516 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:28.468444 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xtmx4_17ac8a3e-a3af-47f6-b21e-9935be5f72d1/limitador/0.log" Jul 07 12:10:29.027868 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:29.027833 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b78z9_3e034319-6cc8-44a2-b487-b6e05a3927b6/discovery/0.log" Jul 07 12:10:30.063715 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:30.063685 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-g2jqz_0a95a75e-3af2-4180-a0c5-c24b9b9a11f1/main/0.log" Jul 07 12:10:30.545547 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:30.545492 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6mkmq_60f314e0-3447-4a36-bf24-6dfce019b8fc/main/0.log" Jul 07 12:10:37.760389 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:37.760350 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6kjkj_0eafca62-9210-42e4-8af3-15faf4201332/global-pull-secret-syncer/0.log" Jul 07 12:10:37.959095 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:37.959054 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v9wwc_9d6ea786-e03d-4f74-a080-914765d20aef/konnectivity-agent/0.log" Jul 07 12:10:38.026864 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:38.026741 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-127.ec2.internal_85d1b63fc295b631cf34855fb24fcee0/haproxy/0.log" Jul 07 12:10:43.307007 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:43.306970 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xtmx4_17ac8a3e-a3af-47f6-b21e-9935be5f72d1/limitador/0.log" Jul 07 12:10:45.220683 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:45.220650 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7b45869b55-748t5_302cdb48-badf-49ef-9c47-5c83ea33cd9b/metrics-server/0.log" Jul 07 12:10:45.394908 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:45.394877 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gq77g_87fd6ff0-9ee3-4be5-8f10-668b5e330994/node-exporter/0.log" Jul 07 12:10:45.415185 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:45.415162 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gq77g_87fd6ff0-9ee3-4be5-8f10-668b5e330994/kube-rbac-proxy/0.log" Jul 07 12:10:45.438099 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:45.438020 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gq77g_87fd6ff0-9ee3-4be5-8f10-668b5e330994/init-textfile/0.log" Jul 07 12:10:45.618951 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:45.618910 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ad412cc7-72f2-4e70-a3d6-a6612bf07caa/prometheus/0.log" Jul 07 12:10:45.639700 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:45.639676 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ad412cc7-72f2-4e70-a3d6-a6612bf07caa/config-reloader/0.log" Jul 07 12:10:45.661925 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:45.661900 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ad412cc7-72f2-4e70-a3d6-a6612bf07caa/thanos-sidecar/0.log" Jul 07 12:10:45.682784 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:45.682749 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ad412cc7-72f2-4e70-a3d6-a6612bf07caa/kube-rbac-proxy-web/0.log" Jul 07 12:10:45.702802 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:45.702699 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ad412cc7-72f2-4e70-a3d6-a6612bf07caa/kube-rbac-proxy/0.log" Jul 07 12:10:45.725184 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:45.725160 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ad412cc7-72f2-4e70-a3d6-a6612bf07caa/kube-rbac-proxy-thanos/0.log" Jul 07 12:10:45.755107 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:45.755060 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ad412cc7-72f2-4e70-a3d6-a6612bf07caa/init-config-reloader/0.log" Jul 07 12:10:48.425115 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.425075 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv"] Jul 07 12:10:48.425629 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.425482 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f92f4576-0839-45e2-9b97-4f4e481828c0" containerName="maas-api" Jul 07 12:10:48.425629 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.425503 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f92f4576-0839-45e2-9b97-4f4e481828c0" containerName="maas-api" Jul 07 12:10:48.425629 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.425564 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39ee4fb-c2f2-48fb-99a9-9728ab3e9358" containerName="maas-api" Jul 07 12:10:48.425629 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.425573 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39ee4fb-c2f2-48fb-99a9-9728ab3e9358" containerName="maas-api" Jul 07 12:10:48.425843 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.425647 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f92f4576-0839-45e2-9b97-4f4e481828c0" containerName="maas-api" Jul 07 12:10:48.425843 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.425663 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e39ee4fb-c2f2-48fb-99a9-9728ab3e9358" containerName="maas-api" Jul 07 12:10:48.428900 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.428873 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.431876 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.431613 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vbjbt\"/\"default-dockercfg-x8ms6\"" Jul 07 12:10:48.431876 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.431613 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vbjbt\"/\"openshift-service-ca.crt\"" Jul 07 12:10:48.431876 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.431765 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vbjbt\"/\"kube-root-ca.crt\"" Jul 07 12:10:48.437610 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.437585 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv"] Jul 07 12:10:48.576705 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.576662 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb117e80-40b4-426e-b0e3-3782736ca8ad-podres\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.576904 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.576738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb117e80-40b4-426e-b0e3-3782736ca8ad-proc\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.576904 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.576768 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngsr5\" (UniqueName: \"kubernetes.io/projected/fb117e80-40b4-426e-b0e3-3782736ca8ad-kube-api-access-ngsr5\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.576904 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.576812 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb117e80-40b4-426e-b0e3-3782736ca8ad-lib-modules\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.576904 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.576838 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb117e80-40b4-426e-b0e3-3782736ca8ad-sys\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.600365 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.600339 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6bb858c6d6-zjt42_54de5cb3-dd83-4415-be7e-711c40fe259d/console/0.log" Jul 07 12:10:48.677619 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.677498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb117e80-40b4-426e-b0e3-3782736ca8ad-lib-modules\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.677619 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.677559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb117e80-40b4-426e-b0e3-3782736ca8ad-sys\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.677619 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.677604 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb117e80-40b4-426e-b0e3-3782736ca8ad-podres\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.677876 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.677644 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb117e80-40b4-426e-b0e3-3782736ca8ad-proc\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.677876 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.677666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngsr5\" (UniqueName: \"kubernetes.io/projected/fb117e80-40b4-426e-b0e3-3782736ca8ad-kube-api-access-ngsr5\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.677876 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.677696 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb117e80-40b4-426e-b0e3-3782736ca8ad-lib-modules\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.677876 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.677695 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb117e80-40b4-426e-b0e3-3782736ca8ad-sys\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.677876 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.677738 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb117e80-40b4-426e-b0e3-3782736ca8ad-proc\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.677876 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.677761 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb117e80-40b4-426e-b0e3-3782736ca8ad-podres\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.686465 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.686436 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngsr5\" (UniqueName: \"kubernetes.io/projected/fb117e80-40b4-426e-b0e3-3782736ca8ad-kube-api-access-ngsr5\") pod \"perf-node-gather-daemonset-dshqv\" (UID: \"fb117e80-40b4-426e-b0e3-3782736ca8ad\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.739858 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.739821 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:48.866918 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:48.866893 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv"] Jul 07 12:10:48.868783 ip-10-0-139-127 kubenswrapper[2578]: W0707 12:10:48.868754 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfb117e80_40b4_426e_b0e3_3782736ca8ad.slice/crio-492ff23e1f849e01b376d4809bb6a7a6386195a0c7f161fb4cb50131c3812e45 WatchSource:0}: Error finding container 492ff23e1f849e01b376d4809bb6a7a6386195a0c7f161fb4cb50131c3812e45: Status 404 returned error can't find the container with id 492ff23e1f849e01b376d4809bb6a7a6386195a0c7f161fb4cb50131c3812e45 Jul 07 12:10:49.195981 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:49.195945 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" event={"ID":"fb117e80-40b4-426e-b0e3-3782736ca8ad","Type":"ContainerStarted","Data":"d68da3f648749df625fe36a9409f87092aee2ba28ab01f22ee0dbe4de117d7ec"} Jul 07 12:10:49.196162 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:49.195993 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" event={"ID":"fb117e80-40b4-426e-b0e3-3782736ca8ad","Type":"ContainerStarted","Data":"492ff23e1f849e01b376d4809bb6a7a6386195a0c7f161fb4cb50131c3812e45"} Jul 07 12:10:49.196162 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:49.196023 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:49.216031 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:49.215919 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" podStartSLOduration=1.215901486 podStartE2EDuration="1.215901486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:10:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:10:49.214149274 +0000 UTC m=+2623.818486136" watchObservedRunningTime="2026-07-07 12:10:49.215901486 +0000 UTC m=+2623.820238326" Jul 07 12:10:50.415249 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:50.415216 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5g7lc_b8a7a959-d967-4b84-b98e-3cc699102bf5/dns/0.log" Jul 07 12:10:50.436282 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:50.436251 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5g7lc_b8a7a959-d967-4b84-b98e-3cc699102bf5/kube-rbac-proxy/0.log" Jul 07 12:10:50.582436 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:50.582409 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nqrsk_ba1d1827-3e47-4686-8173-7c4619849f84/dns-node-resolver/0.log" Jul 07 12:10:51.231990 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:51.231962 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rnzn8_a8319a2e-d262-43be-b188-7d50ac4598be/node-ca/0.log" Jul 07 12:10:52.321344 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:52.321311 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b78z9_3e034319-6cc8-44a2-b487-b6e05a3927b6/discovery/0.log" Jul 07 12:10:53.282422 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:53.282397 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fvpk7_6b00dac5-b502-4544-a725-27b6c15b0639/serve-healthcheck-canary/0.log" Jul 07 12:10:53.855962 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:53.855932 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nbkgr_9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441/kube-rbac-proxy/0.log" Jul 07 12:10:53.877551 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:53.877513 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nbkgr_9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441/exporter/0.log" Jul 07 12:10:53.899085 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:53.899064 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nbkgr_9fc4fbc8-0d3c-497a-a4e8-5d1b5f623441/extractor/0.log" Jul 07 12:10:55.210043 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:55.210014 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-dshqv" Jul 07 12:10:56.702792 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:56.702756 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f68d9489b-gfv5k_6c48f684-542f-438a-91a7-d924a8027930/manager/0.log" Jul 07 12:10:56.719629 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:56.719602 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f68d9489b-gfv5k_6c48f684-542f-438a-91a7-d924a8027930/copy-manifests/0.log" Jul 07 12:10:58.330298 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:58.330219 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-69986dd77b-6tp27_594d0f69-d382-42e6-aab9-e2d7732622b5/manager/0.log" Jul 07 12:10:59.074676 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:10:59.074644 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b78z9_3e034319-6cc8-44a2-b487-b6e05a3927b6/discovery/0.log" Jul 07 12:11:00.565226 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:11:00.565196 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b78z9_3e034319-6cc8-44a2-b487-b6e05a3927b6/discovery/0.log" Jul 07 12:11:04.918509 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:11:04.918482 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5kl2h_674e54c4-6cb6-4fbc-ba4e-743e240039a7/kube-multus-additional-cni-plugins/0.log" Jul 07 12:11:04.939997 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:11:04.939972 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5kl2h_674e54c4-6cb6-4fbc-ba4e-743e240039a7/egress-router-binary-copy/0.log" Jul 07 12:11:04.960582 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:11:04.960558 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5kl2h_674e54c4-6cb6-4fbc-ba4e-743e240039a7/cni-plugins/0.log" Jul 07 12:11:04.980527 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:11:04.980506 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5kl2h_674e54c4-6cb6-4fbc-ba4e-743e240039a7/bond-cni-plugin/0.log" Jul 07 12:11:05.002429 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:11:05.002406 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5kl2h_674e54c4-6cb6-4fbc-ba4e-743e240039a7/routeoverride-cni/0.log" Jul 07 12:11:05.022526 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:11:05.022505 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5kl2h_674e54c4-6cb6-4fbc-ba4e-743e240039a7/whereabouts-cni-bincopy/0.log" Jul 07 12:11:05.043437 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:11:05.043404 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5kl2h_674e54c4-6cb6-4fbc-ba4e-743e240039a7/whereabouts-cni/0.log" Jul 07 12:11:05.458224 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:11:05.458191 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vbmng_6d9acbc8-5af1-4514-bc47-b8c50a06d5a3/kube-multus/0.log" Jul 07 12:11:05.477590 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:11:05.477565 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9vm97_e825e10d-32d1-41d4-aa12-1d1f1dbbb429/network-metrics-daemon/0.log" Jul 07 12:11:05.497063 ip-10-0-139-127 kubenswrapper[2578]: I0707 12:11:05.497038 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9vm97_e825e10d-32d1-41d4-aa12-1d1f1dbbb429/kube-rbac-proxy/0.log"