Jul 01 19:09:23.528115 ip-10-0-142-98 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 01 19:09:23.528127 ip-10-0-142-98 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 01 19:09:23.528133 ip-10-0-142-98 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 01 19:09:23.528391 ip-10-0-142-98 systemd[1]: Failed to start Kubernetes Kubelet. Jul 01 19:09:33.602192 ip-10-0-142-98 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 01 19:09:33.602207 ip-10-0-142-98 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c62a0dc1699543b596893ed6e31e0f1f -- Jul 01 19:11:36.641445 ip-10-0-142-98 systemd[1]: Starting Kubernetes Kubelet... Jul 01 19:11:37.138757 ip-10-0-142-98 systemd[1]: Started Kubernetes Kubelet. Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.056932 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.063981 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.063995 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064000 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064004 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064007 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064010 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064013 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 19:11:37.152744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064016 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064019 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064022 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064026 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064031 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064034 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064037 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064040 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064043 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064046 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064048 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064051 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064054 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064056 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064059 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064062 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064064 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064067 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064069 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064072 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 19:11:37.154861 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064075 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064078 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064080 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064083 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064087 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064090 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064093 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064095 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064098 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064100 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064103 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064106 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064108 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064129 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064133 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064135 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064139 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064141 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064144 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064147 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 19:11:37.155698 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064150 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064152 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064155 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064157 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064160 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064163 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064166 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064168 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064171 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064173 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064176 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064181 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064185 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064188 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064191 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064193 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064196 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064199 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064201 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 19:11:37.158176 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064204 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064207 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064209 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064212 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064215 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064217 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064220 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064250 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064254 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064257 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064259 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064263 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064265 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064268 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064271 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064274 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064276 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064280 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064283 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064285 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 19:11:37.159070 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064288 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064291 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064294 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064296 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064299 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064302 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064304 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064307 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064311 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064315 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064317 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064320 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.064323 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065028 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065035 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065038 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065041 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065044 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065046 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065051 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 19:11:37.161389 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065054 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065057 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065060 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065062 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065065 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065068 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065071 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065074 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065076 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065079 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065082 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065085 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065088 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065090 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065093 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065096 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065100 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065105 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065108 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 19:11:37.162425 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065128 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065132 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065135 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065139 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065141 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065145 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065148 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065151 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065154 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065157 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065160 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065163 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065165 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065168 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065171 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065173 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065176 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065178 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065182 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065184 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 19:11:37.165669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065187 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065189 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065192 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065194 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065198 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065201 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065203 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065206 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065209 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065211 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065214 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065217 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065220 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065222 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065225 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065228 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065230 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065233 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065237 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065241 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 19:11:37.166527 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065244 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065249 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065251 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065254 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065257 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065260 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065262 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065265 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065267 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065270 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065273 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065275 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065278 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065280 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065283 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065285 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065288 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065292 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065295 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065298 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 19:11:37.168081 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065300 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065303 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065306 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065308 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065311 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065314 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065318 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065321 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065324 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065326 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065329 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065331 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065334 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065702 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065707 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065711 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065716 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065719 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065723 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 19:11:37.168868 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065727 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065730 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065733 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065735 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065738 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065741 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065743 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065746 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065749 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065752 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065755 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065757 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065760 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065763 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065765 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065769 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065771 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065774 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065777 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065779 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 19:11:37.169655 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065782 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065785 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065787 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065790 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065792 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065795 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065798 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065800 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065803 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065805 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065808 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065811 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065814 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065817 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065819 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065822 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065824 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065827 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065830 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065833 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 19:11:37.170269 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065835 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065838 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065841 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065843 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065846 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065849 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065851 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065855 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065857 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065860 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065863 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065866 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065868 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065871 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065874 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065876 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065879 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065881 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065884 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065886 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 19:11:37.170952 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065889 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065892 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065894 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065897 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065900 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065902 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065904 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065907 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065909 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065912 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065914 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065917 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065919 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065922 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065924 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065927 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065930 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065934 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065937 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065939 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065942 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 19:11:37.171587 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065945 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065948 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065950 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065954 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065957 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065960 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065962 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065965 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065970 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065973 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065976 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.065978 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066045 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066052 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066059 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066074 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066080 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066084 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066089 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066093 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 01 19:11:37.337944 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066096 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066099 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066103 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066106 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066127 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066133 2577 flags.go:64] FLAG: --cgroup-root="" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066136 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066139 2577 flags.go:64] FLAG: --client-ca-file="" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066143 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066146 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066151 2577 flags.go:64] FLAG: --cluster-domain="" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066154 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066157 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066160 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066164 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066168 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066171 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066173 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066177 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066179 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066184 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066187 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066191 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066194 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 01 19:11:37.352591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066198 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066201 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066204 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066207 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066210 2577 flags.go:64] FLAG: --enable-server="true" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066213 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066217 2577 flags.go:64] FLAG: --event-burst="100" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066220 2577 flags.go:64] FLAG: --event-qps="50" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066223 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066226 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066228 2577 flags.go:64] FLAG: --eviction-hard="" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066232 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066235 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066239 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066242 2577 flags.go:64] FLAG: --eviction-soft="" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066245 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066248 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066251 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066254 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066257 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066260 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066263 2577 flags.go:64] FLAG: --feature-gates="" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066267 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066270 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066273 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 01 19:11:37.355641 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066276 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066279 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066282 2577 flags.go:64] FLAG: --help="false" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066287 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066290 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066293 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066296 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066299 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066303 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066306 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066309 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066311 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066314 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066317 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066320 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066323 2577 flags.go:64] FLAG: --kube-reserved="" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066326 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066329 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066332 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066335 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066340 2577 flags.go:64] FLAG: --lock-file="" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066342 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066346 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066349 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 01 19:11:37.356718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066354 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066357 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066360 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066363 2577 flags.go:64] FLAG: --logging-format="text" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066365 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066369 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066371 2577 flags.go:64] FLAG: --manifest-url="" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066374 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066379 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066382 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066386 2577 flags.go:64] FLAG: --max-pods="110" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066391 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066393 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066396 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066399 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066402 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066405 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066408 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066415 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066418 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066421 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066424 2577 flags.go:64] FLAG: --pod-cidr="" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066427 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 01 19:11:37.357839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066433 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066436 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066439 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066442 2577 flags.go:64] FLAG: --port="10250" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066445 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066449 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ee9d52ee913db9ad" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066452 2577 flags.go:64] FLAG: --qos-reserved="" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066455 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066458 2577 flags.go:64] FLAG: --register-node="true" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066461 2577 flags.go:64] FLAG: --register-with-taints="" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066464 2577 flags.go:64] FLAG: --registry-burst="10" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066470 2577 flags.go:64] FLAG: --registry-qps="5" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066473 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066476 2577 flags.go:64] FLAG: --reserved-memory="" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066480 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066483 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066486 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066490 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066492 2577 flags.go:64] FLAG: --runonce="false" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066495 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066498 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066503 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066506 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066509 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066512 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066515 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 01 19:11:37.358660 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066518 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066521 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066524 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066527 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066529 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066532 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066535 2577 flags.go:64] FLAG: --system-cgroups="" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066538 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066543 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066546 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066549 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066554 2577 flags.go:64] FLAG: --tls-min-version="" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066557 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066560 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066563 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066566 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066568 2577 flags.go:64] FLAG: --v="2" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066573 2577 flags.go:64] FLAG: --version="false" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066577 2577 flags.go:64] FLAG: --vmodule="" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066580 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.066584 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066694 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066698 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066701 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 19:11:37.359635 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066704 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066707 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066710 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066714 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066716 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066719 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066722 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066724 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066727 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066730 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066734 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066737 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066741 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066745 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066748 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066751 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066754 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066757 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066760 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 19:11:37.360390 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066765 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066768 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066771 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066773 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066776 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066778 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066781 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066784 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066788 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066790 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066793 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066796 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066798 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066801 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066803 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066805 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066809 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066812 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066815 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066817 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 19:11:37.361295 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066820 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066822 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066825 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066827 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066830 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066832 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066835 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066838 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066840 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066843 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066845 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066848 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066852 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066855 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066857 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066860 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066862 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066865 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066868 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066870 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 19:11:37.361863 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066873 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066875 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066878 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066881 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066883 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066886 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066889 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066891 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066895 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066897 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066900 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066904 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066908 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066910 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066913 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066916 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066919 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066921 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066924 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 19:11:37.362798 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066926 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066929 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066932 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066934 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066937 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066942 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066945 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066947 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066950 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066953 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066955 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066958 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066960 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066963 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066966 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066969 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066971 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 19:11:37.363364 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.066974 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.067491 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.074342 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.074360 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.075193 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.075200 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075263 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075270 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075274 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075278 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075280 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075284 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075287 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075289 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 19:11:37.363873 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075293 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075295 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075298 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075301 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075303 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075307 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075309 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075312 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075314 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075317 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075320 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075323 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075325 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075328 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075330 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075333 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075335 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075338 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075340 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075343 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 19:11:37.571470 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075347 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075356 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075359 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075361 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075364 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075366 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075370 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075372 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075375 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075377 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075380 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075383 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075385 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075388 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075391 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075393 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075396 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075399 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075402 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075404 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 19:11:37.572477 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075407 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075410 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075412 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075416 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075421 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075423 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075426 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075429 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075431 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075434 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075436 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075441 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075444 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075448 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075457 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075461 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075463 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075466 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075469 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 19:11:37.573069 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075472 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075475 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075477 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075480 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075483 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075485 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075488 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075491 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075493 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075496 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075498 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075501 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075503 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075506 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075509 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075511 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075514 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075517 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075519 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075522 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 19:11:37.573744 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075524 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075527 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075530 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075532 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075536 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075540 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075543 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075546 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075554 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075557 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075559 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075562 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.075567 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075713 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075717 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 19:11:37.574323 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075720 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075723 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075726 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075728 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075731 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075734 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075736 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075739 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075741 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075744 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075747 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075749 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075752 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075754 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075757 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075759 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075762 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075764 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075767 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075770 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 19:11:37.574969 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075772 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075775 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075780 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075782 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075785 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075788 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075796 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075799 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075802 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075805 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075808 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075810 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075814 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075818 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075821 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075824 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075827 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075830 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075833 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 19:11:37.575558 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075836 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075839 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075843 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075846 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075849 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075851 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075854 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075857 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075859 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075862 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075864 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075867 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075870 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075872 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075875 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075880 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075882 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075885 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075893 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075896 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 19:11:37.652381 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075904 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075907 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075910 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075912 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075915 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075918 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075920 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075923 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075925 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075928 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075931 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075934 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075937 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075939 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075942 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075945 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075947 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075950 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075952 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075955 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 19:11:37.653015 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075958 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075960 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075963 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075965 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075968 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075971 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075973 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075979 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075982 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075984 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075987 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075990 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.075992 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.076000 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.076003 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.076006 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.076008 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 19:11:37.657961 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:37.076011 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 19:11:37.658818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.076016 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 19:11:37.658818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.076134 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 01 19:11:37.658818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.078204 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 01 19:11:37.658818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.079613 2577 server.go:1019] "Starting client certificate rotation" Jul 01 19:11:37.658818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.079709 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 19:11:37.658818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.080609 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 19:11:37.658818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.082380 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 19:11:37.658818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.084658 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 19:11:37.658818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.100181 2577 log.go:25] "Validated CRI v1 runtime API" Jul 01 19:11:37.658818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.102245 2577 log.go:25] "Validated CRI v1 image API" Jul 01 19:11:37.658818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.103827 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 01 19:11:37.658818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.110069 2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b520c6f3-6ca8-4463-b86d-5be354f893b0:/dev/nvme0n1p3 d6f32a5a-f73f-48f0-b7a5-1a84da19660e:/dev/nvme0n1p4] Jul 01 19:11:37.659261 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.110084 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 01 19:11:37.659891 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.116124 2577 manager.go:216] Machine: {Timestamp:2026-07-01 19:11:37.113816043 +0000 UTC m=+0.341990589 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3090832 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ceb0f84390ca17d323a4a6975c280 SystemUUID:ec2ceb0f-8439-0ca1-7d32-3a4a6975c280 BootID:c62a0dc1-6995-43b5-9689-3ed6e31e0f1f Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7c:21:16:95:1f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7c:21:16:95:1f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:7f:69:dd:13:3f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 01 19:11:37.659891 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.116261 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 01 19:11:37.659891 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.116343 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 01 19:11:37.659891 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.120345 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 01 19:11:37.659891 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.120379 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-98.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 01 19:11:37.659891 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.121923 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 01 19:11:37.659891 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.121939 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 01 19:11:37.659891 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.121947 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.122690 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.123567 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.125481 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.126524 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.126637 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.126663 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.126671 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.126686 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.126695 2577 kubelet.go:398] "Adding apiserver pod source" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.126704 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.127135 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.127153 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.129838 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.131516 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.131544 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.131690 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.131703 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.131708 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.131714 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.131722 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.131728 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.132331 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.132338 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.132345 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.132351 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.132360 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.132368 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.134389 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.134512 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.137752 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-98.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.137801 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.137898 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-98.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.138177 2577 server.go:1268] "Started kubelet" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.138497 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.139400 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.139438 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.140457 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.141414 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 01 19:11:37.660342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.147069 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.147080 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.147195 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-98.ec2.internal\" not found" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.147855 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.147869 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.147948 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.147959 2577 factory.go:55] Registering systemd factory Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.147975 2577 factory.go:223] Registration of the systemd container factory successfully Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.147981 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.147962 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.147998 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-txwcb" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.148178 2577 factory.go:153] Registering CRI-O factory Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.148193 2577 factory.go:223] Registration of the crio container factory successfully Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.148240 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.148266 2577 factory.go:103] Registering Raw factory Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.148279 2577 manager.go:1196] Started watching for new ooms in manager Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.148753 2577 manager.go:349] Starting recovery of all containers Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.147240 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-98.ec2.internal.18be40349934dd3c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-98.ec2.internal,UID:ip-10-0-142-98.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-98.ec2.internal,},FirstTimestamp:2026-07-01 19:11:37.138150716 +0000 UTC m=+0.366325261,LastTimestamp:2026-07-01 19:11:37.138150716 +0000 UTC m=+0.366325261,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-98.ec2.internal,}" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.151751 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.154653 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.154685 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-98.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.164001 2577 manager.go:354] Recovery completed Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.167911 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-txwcb" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.168554 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.171549 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.171573 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.171582 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.172007 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.172018 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.172036 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.172019 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.173184 2577 policy_none.go:49] "None policy: Start" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.173198 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 01 19:11:37.661830 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.173211 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.173321 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-98.ec2.internal.18be40349b32a82f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-98.ec2.internal,UID:ip-10-0-142-98.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-98.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-98.ec2.internal,},FirstTimestamp:2026-07-01 19:11:37.171560495 +0000 UTC m=+0.399735040,LastTimestamp:2026-07-01 19:11:37.171560495 +0000 UTC m=+0.399735040,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-98.ec2.internal,}" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.175291 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.175308 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.175317 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.179038 2577 policy_none.go:47] "Start" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.219778 2577 manager.go:337] "Starting Device Plugin manager" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.219805 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.219815 2577 server.go:87] "Starting device plugin registration server" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.220063 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.220074 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.220201 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.220274 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.220283 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.220939 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.220985 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-98.ec2.internal\" not found" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.244344 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.245431 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.245463 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.245482 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.245512 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.247848 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.320771 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.321668 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.321696 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.321709 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.321736 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.329833 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.329851 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-98.ec2.internal\": node \"ip-10-0-142-98.ec2.internal\" not found" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.346026 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-98.ec2.internal"] Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.346081 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:11:37.662898 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.346848 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.346879 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.346896 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.348572 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.348675 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.348703 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.349295 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.349323 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.349333 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.349295 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.349403 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.349414 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.351075 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.351095 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.351787 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.351819 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.351834 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.358414 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-98.ec2.internal\" not found" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.366630 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-98.ec2.internal\" not found" node="ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.371852 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-98.ec2.internal\" not found" node="ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.448273 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ae41eef750adfd1eb4d5837b9a03cd1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal\" (UID: \"8ae41eef750adfd1eb4d5837b9a03cd1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.448298 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ae41eef750adfd1eb4d5837b9a03cd1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal\" (UID: \"8ae41eef750adfd1eb4d5837b9a03cd1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.458857 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-98.ec2.internal\" not found" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.548506 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ae41eef750adfd1eb4d5837b9a03cd1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal\" (UID: \"8ae41eef750adfd1eb4d5837b9a03cd1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.664160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.548530 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ae41eef750adfd1eb4d5837b9a03cd1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal\" (UID: \"8ae41eef750adfd1eb4d5837b9a03cd1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.665683 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.548603 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ae41eef750adfd1eb4d5837b9a03cd1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal\" (UID: \"8ae41eef750adfd1eb4d5837b9a03cd1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.665683 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.548887 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8247c4c3a47d291912a129baeb7becc3-config\") pod \"kube-apiserver-proxy-ip-10-0-142-98.ec2.internal\" (UID: \"8247c4c3a47d291912a129baeb7becc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.665683 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.548895 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ae41eef750adfd1eb4d5837b9a03cd1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal\" (UID: \"8ae41eef750adfd1eb4d5837b9a03cd1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.665683 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.559072 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-98.ec2.internal\" not found" Jul 01 19:11:37.665683 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.649769 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8247c4c3a47d291912a129baeb7becc3-config\") pod \"kube-apiserver-proxy-ip-10-0-142-98.ec2.internal\" (UID: \"8247c4c3a47d291912a129baeb7becc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.665683 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.649843 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8247c4c3a47d291912a129baeb7becc3-config\") pod \"kube-apiserver-proxy-ip-10-0-142-98.ec2.internal\" (UID: \"8247c4c3a47d291912a129baeb7becc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.665683 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.659813 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-98.ec2.internal\" not found" Jul 01 19:11:37.669974 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.669957 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.674986 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.674967 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-98.ec2.internal" Jul 01 19:11:37.759945 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.759916 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-98.ec2.internal\" not found" Jul 01 19:11:37.860390 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.860368 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-98.ec2.internal\" not found" Jul 01 19:11:37.960959 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:37.960881 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-98.ec2.internal\" not found" Jul 01 19:11:37.998520 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:37.998500 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:11:38.062026 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:38.061896 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-98.ec2.internal\" not found" Jul 01 19:11:38.079060 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.079038 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 01 19:11:38.079194 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.079172 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 19:11:38.079270 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.079193 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 19:11:38.162424 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:38.162396 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-98.ec2.internal\" not found" Jul 01 19:11:38.172572 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.172533 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 19:06:37 +0000 UTC" deadline="2028-03-01 15:17:46.161977486 +0000 UTC" Jul 01 19:11:38.172572 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.172574 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14612h6m7.989410893s" Jul 01 19:11:38.174227 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:38.174203 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8247c4c3a47d291912a129baeb7becc3.slice/crio-8e90b42e58618a35466caaa6c43676a2640bf060e41903786b421fb68ca73118 WatchSource:0}: Error finding container 8e90b42e58618a35466caaa6c43676a2640bf060e41903786b421fb68ca73118: Status 404 returned error can't find the container with id 8e90b42e58618a35466caaa6c43676a2640bf060e41903786b421fb68ca73118 Jul 01 19:11:38.179938 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:38.179913 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ae41eef750adfd1eb4d5837b9a03cd1.slice/crio-88c29c0378c2d814810e6cafa2a5aae04b7d17fba0c47b581c24d3ecb109c76e WatchSource:0}: Error finding container 88c29c0378c2d814810e6cafa2a5aae04b7d17fba0c47b581c24d3ecb109c76e: Status 404 returned error can't find the container with id 88c29c0378c2d814810e6cafa2a5aae04b7d17fba0c47b581c24d3ecb109c76e Jul 01 19:11:38.181579 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.181569 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 19:11:38.185750 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.185733 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:11:38.233310 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.233251 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:11:38.247361 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.247334 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" Jul 01 19:11:38.249020 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.248983 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" event={"ID":"8ae41eef750adfd1eb4d5837b9a03cd1","Type":"ContainerStarted","Data":"88c29c0378c2d814810e6cafa2a5aae04b7d17fba0c47b581c24d3ecb109c76e"} Jul 01 19:11:38.249987 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.249969 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-98.ec2.internal" event={"ID":"8247c4c3a47d291912a129baeb7becc3","Type":"ContainerStarted","Data":"8e90b42e58618a35466caaa6c43676a2640bf060e41903786b421fb68ca73118"} Jul 01 19:11:38.259300 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.259283 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 19:11:38.260595 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.260584 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-98.ec2.internal" Jul 01 19:11:38.267812 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.267798 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 19:11:38.952635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:38.952600 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:11:39.127442 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.127373 2577 apiserver.go:52] "Watching apiserver" Jul 01 19:11:39.132761 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.132723 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 01 19:11:39.134328 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.134298 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-pl6zn","kube-system/konnectivity-agent-cqtzm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg","openshift-cluster-node-tuning-operator/tuned-lmm5r","openshift-dns/node-resolver-7tp8j","openshift-network-operator/iptables-alerter-x2scp","openshift-ovn-kubernetes/ovnkube-node-ksmnn","kube-system/global-pull-secret-syncer-66xvf","kube-system/kube-apiserver-proxy-ip-10-0-142-98.ec2.internal","openshift-image-registry/node-ca-bmzq6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal","openshift-multus/multus-24dr2","openshift-multus/multus-additional-cni-plugins-xlfr9","openshift-multus/network-metrics-daemon-cbmfg"] Jul 01 19:11:39.136146 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.136108 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.137459 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.137439 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.138886 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.138681 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 01 19:11:39.138886 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.138682 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 01 19:11:39.138886 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.138794 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 01 19:11:39.139098 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.139032 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.139299 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.139277 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 01 19:11:39.140690 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.139690 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 01 19:11:39.140690 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.140214 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 01 19:11:39.140690 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.140235 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nhblg\"" Jul 01 19:11:39.142127 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.141266 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 01 19:11:39.142127 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.141513 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 01 19:11:39.142127 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.141749 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 01 19:11:39.142127 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.141962 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 01 19:11:39.142471 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.142450 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 01 19:11:39.142547 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.142528 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-728cj\"" Jul 01 19:11:39.142713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.142452 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.143825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.143640 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 01 19:11:39.143825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.143660 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 01 19:11:39.143825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.143696 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v56zt\"" Jul 01 19:11:39.143825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.143712 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 01 19:11:39.144601 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.144176 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7tp8j" Jul 01 19:11:39.147030 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.146039 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 01 19:11:39.147030 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.146119 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 01 19:11:39.147030 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.146494 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 01 19:11:39.147030 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.146498 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 01 19:11:39.147030 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.146559 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-z9k5p\"" Jul 01 19:11:39.147030 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.146669 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5qntm\"" Jul 01 19:11:39.147030 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.146741 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x2scp" Jul 01 19:11:39.148279 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.148224 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 01 19:11:39.148517 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.148498 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:39.148604 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.148575 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 01 19:11:39.148689 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.148570 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:11:39.149053 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.149031 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-f5fhf\"" Jul 01 19:11:39.149162 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.149083 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 01 19:11:39.149711 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.149556 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 01 19:11:39.150539 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.150518 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:39.150648 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.150592 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:11:39.151673 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.151657 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cqtzm" Jul 01 19:11:39.153442 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.153316 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bmzq6" Jul 01 19:11:39.153597 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.153575 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 01 19:11:39.154039 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.154019 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bkv9c\"" Jul 01 19:11:39.154237 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.154055 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 01 19:11:39.154636 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.154621 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:39.154723 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.154681 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:11:39.155048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.155029 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 01 19:11:39.155296 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.155274 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 01 19:11:39.155677 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.155655 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 01 19:11:39.156204 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.156022 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wnc97\"" Jul 01 19:11:39.156376 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.156355 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.158104 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158031 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-lib-modules\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.158104 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158078 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.158248 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158109 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-registration-dir\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.158248 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158162 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-tuned\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.158248 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158186 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-etc-openvswitch\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.158248 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158239 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/771dc0a4-073a-4c63-85c5-e405dad55641-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.158449 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158280 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-sysctl-d\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.158449 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158306 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-cni-netd\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.158449 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158339 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-socket-dir\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.158449 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158362 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-etc-selinux\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.158449 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158431 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/771dc0a4-073a-4c63-85c5-e405dad55641-cni-binary-copy\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.158694 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158489 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/771dc0a4-073a-4c63-85c5-e405dad55641-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.158694 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158515 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-systemd\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.158694 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158539 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-host\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.158694 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158564 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr25w\" (UniqueName: \"kubernetes.io/projected/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-kube-api-access-dr25w\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.158694 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158587 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-run-systemd\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.158694 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158679 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-run-openvswitch\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.158952 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158743 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-run-ovn\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.158952 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158768 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-cni-bin\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.158952 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158793 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-ovnkube-config\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.158952 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158820 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.158952 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158853 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-device-dir\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.158952 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158916 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/771dc0a4-073a-4c63-85c5-e405dad55641-cnibin\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.158952 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158945 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/771dc0a4-073a-4c63-85c5-e405dad55641-os-release\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.159284 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.158987 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-var-lib-openvswitch\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.159284 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159015 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-node-log\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.159284 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159035 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-j5ckl\"" Jul 01 19:11:39.159284 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-log-socket\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.159284 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159072 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/771dc0a4-073a-4c63-85c5-e405dad55641-system-cni-dir\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.159284 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159244 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-ovn-node-metrics-cert\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.159645 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159288 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.159645 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159316 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-modprobe-d\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.159645 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159340 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-sysconfig\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.159645 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159362 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-sys\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.159645 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159388 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-systemd-units\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.159645 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159482 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:39.159645 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159507 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/691b723a-898b-4dbc-81ee-6646ef35df0a-tmp-dir\") pod \"node-resolver-7tp8j\" (UID: \"691b723a-898b-4dbc-81ee-6646ef35df0a\") " pod="openshift-dns/node-resolver-7tp8j" Jul 01 19:11:39.159645 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159532 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-env-overrides\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.159645 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159559 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbgwq\" (UniqueName: \"kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq\") pod \"network-check-target-pl6zn\" (UID: \"b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83\") " pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:39.159645 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159598 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/771dc0a4-073a-4c63-85c5-e405dad55641-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.159645 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159624 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-ovnkube-script-lib\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.160171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159658 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvrcr\" (UniqueName: \"kubernetes.io/projected/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-kube-api-access-mvrcr\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.160171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159684 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ab8ff4c5-e79a-47a7-836d-569f0a408205-iptables-alerter-script\") pod \"iptables-alerter-x2scp\" (UID: \"ab8ff4c5-e79a-47a7-836d-569f0a408205\") " pod="openshift-network-operator/iptables-alerter-x2scp" Jul 01 19:11:39.160171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159712 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzpgt\" (UniqueName: \"kubernetes.io/projected/771dc0a4-073a-4c63-85c5-e405dad55641-kube-api-access-hzpgt\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.160171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159739 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-var-lib-kubelet\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.160171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159765 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-tmp\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.160171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159788 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 01 19:11:39.160171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.159790 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-kubernetes\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.160171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.160014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/691b723a-898b-4dbc-81ee-6646ef35df0a-hosts-file\") pod \"node-resolver-7tp8j\" (UID: \"691b723a-898b-4dbc-81ee-6646ef35df0a\") " pod="openshift-dns/node-resolver-7tp8j" Jul 01 19:11:39.160171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.160096 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-kubelet\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.160171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.160125 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fkf9\" (UniqueName: \"kubernetes.io/projected/86f9c330-5a8b-49cc-85fc-b285b09a64cb-kube-api-access-6fkf9\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.160171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.160151 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-sysctl-conf\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.160171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.160172 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2m5x\" (UniqueName: \"kubernetes.io/projected/691b723a-898b-4dbc-81ee-6646ef35df0a-kube-api-access-j2m5x\") pod \"node-resolver-7tp8j\" (UID: \"691b723a-898b-4dbc-81ee-6646ef35df0a\") " pod="openshift-dns/node-resolver-7tp8j" Jul 01 19:11:39.160784 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.160211 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-sys-fs\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.160784 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.160235 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-run\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.160784 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.160275 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-run-netns\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.160784 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.160300 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-slash\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.160784 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.160329 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab8ff4c5-e79a-47a7-836d-569f0a408205-host-slash\") pod \"iptables-alerter-x2scp\" (UID: \"ab8ff4c5-e79a-47a7-836d-569f0a408205\") " pod="openshift-network-operator/iptables-alerter-x2scp" Jul 01 19:11:39.160784 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.160355 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7g2z\" (UniqueName: \"kubernetes.io/projected/ab8ff4c5-e79a-47a7-836d-569f0a408205-kube-api-access-z7g2z\") pod \"iptables-alerter-x2scp\" (UID: \"ab8ff4c5-e79a-47a7-836d-569f0a408205\") " pod="openshift-network-operator/iptables-alerter-x2scp" Jul 01 19:11:39.160784 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.160379 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4b9h\" (UniqueName: \"kubernetes.io/projected/47beecfe-aae0-485b-b9f4-9c75b22096ea-kube-api-access-v4b9h\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:39.161963 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.161933 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 19:11:39.190204 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.190159 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cwpmc" Jul 01 19:11:39.198259 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.198236 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cwpmc" Jul 01 19:11:39.249053 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.249028 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 01 19:11:39.261375 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.261344 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-env-overrides\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.261509 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.261398 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbgwq\" (UniqueName: \"kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq\") pod \"network-check-target-pl6zn\" (UID: \"b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83\") " pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:39.261571 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.261524 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q29v\" (UniqueName: \"kubernetes.io/projected/99b873f3-650c-4b66-acf7-5eb80ecb4c70-kube-api-access-5q29v\") pod \"node-ca-bmzq6\" (UID: \"99b873f3-650c-4b66-acf7-5eb80ecb4c70\") " pod="openshift-image-registry/node-ca-bmzq6" Jul 01 19:11:39.261571 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.261554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/771dc0a4-073a-4c63-85c5-e405dad55641-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.261685 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.261583 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-system-cni-dir\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.261685 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.261607 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-multus-cni-dir\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.261685 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.261655 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-ovnkube-script-lib\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.261846 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.261691 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvrcr\" (UniqueName: \"kubernetes.io/projected/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-kube-api-access-mvrcr\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.261846 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.261720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ab8ff4c5-e79a-47a7-836d-569f0a408205-iptables-alerter-script\") pod \"iptables-alerter-x2scp\" (UID: \"ab8ff4c5-e79a-47a7-836d-569f0a408205\") " pod="openshift-network-operator/iptables-alerter-x2scp" Jul 01 19:11:39.261846 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.261749 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzpgt\" (UniqueName: \"kubernetes.io/projected/771dc0a4-073a-4c63-85c5-e405dad55641-kube-api-access-hzpgt\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.261846 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.261779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-var-lib-kubelet\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.262057 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.261850 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-var-lib-kubelet\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.262057 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262001 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-env-overrides\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.262247 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262095 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-tmp\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.262247 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262129 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05fffc12-7ff5-4ae0-9f3d-85267bb7eaef-agent-certs\") pod \"konnectivity-agent-cqtzm\" (UID: \"05fffc12-7ff5-4ae0-9f3d-85267bb7eaef\") " pod="kube-system/konnectivity-agent-cqtzm" Jul 01 19:11:39.262247 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262161 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/409fb45f-4128-440a-98ec-58763de6e171-cni-binary-copy\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.262247 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-kubernetes\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.262451 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262250 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/691b723a-898b-4dbc-81ee-6646ef35df0a-hosts-file\") pod \"node-resolver-7tp8j\" (UID: \"691b723a-898b-4dbc-81ee-6646ef35df0a\") " pod="openshift-dns/node-resolver-7tp8j" Jul 01 19:11:39.262451 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262276 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-kubelet\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.262451 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262292 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/771dc0a4-073a-4c63-85c5-e405dad55641-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.262451 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262339 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-kubelet\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.262451 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262357 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fkf9\" (UniqueName: \"kubernetes.io/projected/86f9c330-5a8b-49cc-85fc-b285b09a64cb-kube-api-access-6fkf9\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.262451 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262356 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ab8ff4c5-e79a-47a7-836d-569f0a408205-iptables-alerter-script\") pod \"iptables-alerter-x2scp\" (UID: \"ab8ff4c5-e79a-47a7-836d-569f0a408205\") " pod="openshift-network-operator/iptables-alerter-x2scp" Jul 01 19:11:39.262451 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262385 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/691b723a-898b-4dbc-81ee-6646ef35df0a-hosts-file\") pod \"node-resolver-7tp8j\" (UID: \"691b723a-898b-4dbc-81ee-6646ef35df0a\") " pod="openshift-dns/node-resolver-7tp8j" Jul 01 19:11:39.262451 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262387 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-hostroot\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.262451 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262436 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-kubernetes\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.262451 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262442 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-run-multus-certs\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262474 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99b873f3-650c-4b66-acf7-5eb80ecb4c70-host\") pod \"node-ca-bmzq6\" (UID: \"99b873f3-650c-4b66-acf7-5eb80ecb4c70\") " pod="openshift-image-registry/node-ca-bmzq6" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262506 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-sysctl-conf\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262524 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-ovnkube-script-lib\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262534 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2m5x\" (UniqueName: \"kubernetes.io/projected/691b723a-898b-4dbc-81ee-6646ef35df0a-kube-api-access-j2m5x\") pod \"node-resolver-7tp8j\" (UID: \"691b723a-898b-4dbc-81ee-6646ef35df0a\") " pod="openshift-dns/node-resolver-7tp8j" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262569 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sxdz\" (UniqueName: \"kubernetes.io/projected/409fb45f-4128-440a-98ec-58763de6e171-kube-api-access-4sxdz\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262602 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-sys-fs\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262626 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-run\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262665 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-run-netns\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262691 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-multus-conf-dir\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262702 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-sys-fs\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262717 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-sysctl-conf\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262717 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-slash\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262759 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab8ff4c5-e79a-47a7-836d-569f0a408205-host-slash\") pod \"iptables-alerter-x2scp\" (UID: \"ab8ff4c5-e79a-47a7-836d-569f0a408205\") " pod="openshift-network-operator/iptables-alerter-x2scp" Jul 01 19:11:39.262939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262776 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-run-netns\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262784 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7g2z\" (UniqueName: \"kubernetes.io/projected/ab8ff4c5-e79a-47a7-836d-569f0a408205-kube-api-access-z7g2z\") pod \"iptables-alerter-x2scp\" (UID: \"ab8ff4c5-e79a-47a7-836d-569f0a408205\") " pod="openshift-network-operator/iptables-alerter-x2scp" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262790 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-slash\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262818 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4b9h\" (UniqueName: \"kubernetes.io/projected/47beecfe-aae0-485b-b9f4-9c75b22096ea-kube-api-access-v4b9h\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262815 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-run\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262827 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab8ff4c5-e79a-47a7-836d-569f0a408205-host-slash\") pod \"iptables-alerter-x2scp\" (UID: \"ab8ff4c5-e79a-47a7-836d-569f0a408205\") " pod="openshift-network-operator/iptables-alerter-x2scp" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262941 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-etc-kubernetes\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262948 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.262984 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-lib-modules\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263027 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263047 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-registration-dir\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263082 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-tuned\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263106 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-etc-openvswitch\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263118 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263140 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/771dc0a4-073a-4c63-85c5-e405dad55641-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.263658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-sysctl-d\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263176 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-etc-openvswitch\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263176 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-cni-netd\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263193 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-lib-modules\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263205 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-socket-dir\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263209 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-cni-netd\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263245 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-var-lib-cni-multus\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263277 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-var-lib-kubelet\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263291 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-socket-dir\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263307 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/409fb45f-4128-440a-98ec-58763de6e171-multus-daemon-config\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263337 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-sysctl-d\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263346 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-registration-dir\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263394 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-etc-selinux\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/771dc0a4-073a-4c63-85c5-e405dad55641-cni-binary-copy\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.264324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263472 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-etc-selinux\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263482 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/771dc0a4-073a-4c63-85c5-e405dad55641-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263596 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-systemd\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263628 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-host\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263668 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr25w\" (UniqueName: \"kubernetes.io/projected/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-kube-api-access-dr25w\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263700 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263731 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-run-systemd\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-run-openvswitch\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263774 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/771dc0a4-073a-4c63-85c5-e405dad55641-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263818 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-run-ovn\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263826 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-run-systemd\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263846 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-cni-bin\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263874 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/943fd8e6-36e9-45d0-a60f-78d295d95d9f-dbus\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263904 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-run-openvswitch\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.265048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263904 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-multus-socket-dir-parent\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263872 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-systemd\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263939 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/771dc0a4-073a-4c63-85c5-e405dad55641-cni-binary-copy\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263954 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-cni-bin\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263986 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-run-ovn\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.263993 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-host\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264028 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-run-netns\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/99b873f3-650c-4b66-acf7-5eb80ecb4c70-serviceca\") pod \"node-ca-bmzq6\" (UID: \"99b873f3-650c-4b66-acf7-5eb80ecb4c70\") " pod="openshift-image-registry/node-ca-bmzq6" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264091 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-ovnkube-config\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264098 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/771dc0a4-073a-4c63-85c5-e405dad55641-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264141 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264190 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-device-dir\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264217 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/771dc0a4-073a-4c63-85c5-e405dad55641-cnibin\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264233 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.265825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264242 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/771dc0a4-073a-4c63-85c5-e405dad55641-os-release\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264293 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-var-lib-openvswitch\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264325 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-node-log\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264329 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86f9c330-5a8b-49cc-85fc-b285b09a64cb-device-dir\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264350 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-log-socket\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264357 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/771dc0a4-073a-4c63-85c5-e405dad55641-os-release\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264368 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-var-lib-openvswitch\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264378 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/771dc0a4-073a-4c63-85c5-e405dad55641-system-cni-dir\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264280 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/771dc0a4-073a-4c63-85c5-e405dad55641-cnibin\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264402 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-log-socket\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264408 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-ovn-node-metrics-cert\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264408 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-node-log\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264438 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/771dc0a4-073a-4c63-85c5-e405dad55641-system-cni-dir\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264449 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.266554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264481 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/943fd8e6-36e9-45d0-a60f-78d295d95d9f-kubelet-config\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264507 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-cnibin\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264527 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264552 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-os-release\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264584 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-run-k8s-cni-cncf-io\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264611 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-var-lib-cni-bin\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264611 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-ovnkube-config\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264641 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-modprobe-d\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264668 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-sysconfig\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264692 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-sys\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264717 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-systemd-units\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264743 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264728 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-modprobe-d\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264751 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-sysconfig\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264815 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-systemd-units\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.267333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264823 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-sys\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.268094 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264824 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05fffc12-7ff5-4ae0-9f3d-85267bb7eaef-konnectivity-ca\") pod \"konnectivity-agent-cqtzm\" (UID: \"05fffc12-7ff5-4ae0-9f3d-85267bb7eaef\") " pod="kube-system/konnectivity-agent-cqtzm" Jul 01 19:11:39.268094 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.264864 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/691b723a-898b-4dbc-81ee-6646ef35df0a-tmp-dir\") pod \"node-resolver-7tp8j\" (UID: \"691b723a-898b-4dbc-81ee-6646ef35df0a\") " pod="openshift-dns/node-resolver-7tp8j" Jul 01 19:11:39.268094 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.264915 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:11:39.268094 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.265022 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs podName:47beecfe-aae0-485b-b9f4-9c75b22096ea nodeName:}" failed. No retries permitted until 2026-07-01 19:11:39.765001214 +0000 UTC m=+2.993175771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs") pod "network-metrics-daemon-cbmfg" (UID: "47beecfe-aae0-485b-b9f4-9c75b22096ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:11:39.268094 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.265167 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/691b723a-898b-4dbc-81ee-6646ef35df0a-tmp-dir\") pod \"node-resolver-7tp8j\" (UID: \"691b723a-898b-4dbc-81ee-6646ef35df0a\") " pod="openshift-dns/node-resolver-7tp8j" Jul 01 19:11:39.268094 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.267065 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-etc-tuned\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.268094 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.267116 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-tmp\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.268094 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.267144 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-ovn-node-metrics-cert\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.276464 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.276417 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:11:39.276464 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.276461 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:11:39.276615 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.276477 2577 projected.go:196] Error preparing data for projected volume kube-api-access-sbgwq for pod openshift-network-diagnostics/network-check-target-pl6zn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:11:39.276615 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.276542 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq podName:b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83 nodeName:}" failed. No retries permitted until 2026-07-01 19:11:39.776524212 +0000 UTC m=+3.004698753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sbgwq" (UniqueName: "kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq") pod "network-check-target-pl6zn" (UID: "b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:11:39.279422 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.279357 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzpgt\" (UniqueName: \"kubernetes.io/projected/771dc0a4-073a-4c63-85c5-e405dad55641-kube-api-access-hzpgt\") pod \"multus-additional-cni-plugins-xlfr9\" (UID: \"771dc0a4-073a-4c63-85c5-e405dad55641\") " pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.279422 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.279399 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fkf9\" (UniqueName: \"kubernetes.io/projected/86f9c330-5a8b-49cc-85fc-b285b09a64cb-kube-api-access-6fkf9\") pod \"aws-ebs-csi-driver-node-qqqpg\" (UID: \"86f9c330-5a8b-49cc-85fc-b285b09a64cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.279903 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.279857 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2m5x\" (UniqueName: \"kubernetes.io/projected/691b723a-898b-4dbc-81ee-6646ef35df0a-kube-api-access-j2m5x\") pod \"node-resolver-7tp8j\" (UID: \"691b723a-898b-4dbc-81ee-6646ef35df0a\") " pod="openshift-dns/node-resolver-7tp8j" Jul 01 19:11:39.280095 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.280068 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7g2z\" (UniqueName: \"kubernetes.io/projected/ab8ff4c5-e79a-47a7-836d-569f0a408205-kube-api-access-z7g2z\") pod \"iptables-alerter-x2scp\" (UID: \"ab8ff4c5-e79a-47a7-836d-569f0a408205\") " pod="openshift-network-operator/iptables-alerter-x2scp" Jul 01 19:11:39.280095 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.280082 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvrcr\" (UniqueName: \"kubernetes.io/projected/f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6-kube-api-access-mvrcr\") pod \"ovnkube-node-ksmnn\" (UID: \"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.280294 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.280089 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4b9h\" (UniqueName: \"kubernetes.io/projected/47beecfe-aae0-485b-b9f4-9c75b22096ea-kube-api-access-v4b9h\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:39.280935 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.280912 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr25w\" (UniqueName: \"kubernetes.io/projected/60d04e1f-c2df-40a5-9c13-d1109fbc7e69-kube-api-access-dr25w\") pod \"tuned-lmm5r\" (UID: \"60d04e1f-c2df-40a5-9c13-d1109fbc7e69\") " pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.366042 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366010 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-hostroot\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366042 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366045 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-run-multus-certs\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366071 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99b873f3-650c-4b66-acf7-5eb80ecb4c70-host\") pod \"node-ca-bmzq6\" (UID: \"99b873f3-650c-4b66-acf7-5eb80ecb4c70\") " pod="openshift-image-registry/node-ca-bmzq6" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366096 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sxdz\" (UniqueName: \"kubernetes.io/projected/409fb45f-4128-440a-98ec-58763de6e171-kube-api-access-4sxdz\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366119 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-multus-conf-dir\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366156 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-etc-kubernetes\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366157 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-hostroot\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366165 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-run-multus-certs\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366183 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99b873f3-650c-4b66-acf7-5eb80ecb4c70-host\") pod \"node-ca-bmzq6\" (UID: \"99b873f3-650c-4b66-acf7-5eb80ecb4c70\") " pod="openshift-image-registry/node-ca-bmzq6" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366193 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-var-lib-cni-multus\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366222 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-multus-conf-dir\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366232 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-etc-kubernetes\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366242 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-var-lib-kubelet\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366264 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/409fb45f-4128-440a-98ec-58763de6e171-multus-daemon-config\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366274 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-var-lib-cni-multus\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366297 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:39.366313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366309 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-var-lib-kubelet\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366326 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/943fd8e6-36e9-45d0-a60f-78d295d95d9f-dbus\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366347 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-multus-socket-dir-parent\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366372 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-run-netns\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366387 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/99b873f3-650c-4b66-acf7-5eb80ecb4c70-serviceca\") pod \"node-ca-bmzq6\" (UID: \"99b873f3-650c-4b66-acf7-5eb80ecb4c70\") " pod="openshift-image-registry/node-ca-bmzq6" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366409 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/943fd8e6-36e9-45d0-a60f-78d295d95d9f-kubelet-config\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-cnibin\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.366435 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366455 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-os-release\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366467 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-run-netns\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366479 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-run-k8s-cni-cncf-io\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.366500 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret podName:943fd8e6-36e9-45d0-a60f-78d295d95d9f nodeName:}" failed. No retries permitted until 2026-07-01 19:11:39.866478084 +0000 UTC m=+3.094652622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret") pod "global-pull-secret-syncer-66xvf" (UID: "943fd8e6-36e9-45d0-a60f-78d295d95d9f") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366498 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/943fd8e6-36e9-45d0-a60f-78d295d95d9f-dbus\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366521 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-run-k8s-cni-cncf-io\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366538 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-var-lib-cni-bin\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366552 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-multus-socket-dir-parent\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366576 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-cnibin\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366598 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-host-var-lib-cni-bin\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366605 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05fffc12-7ff5-4ae0-9f3d-85267bb7eaef-konnectivity-ca\") pod \"konnectivity-agent-cqtzm\" (UID: \"05fffc12-7ff5-4ae0-9f3d-85267bb7eaef\") " pod="kube-system/konnectivity-agent-cqtzm" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366621 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/943fd8e6-36e9-45d0-a60f-78d295d95d9f-kubelet-config\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366647 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q29v\" (UniqueName: \"kubernetes.io/projected/99b873f3-650c-4b66-acf7-5eb80ecb4c70-kube-api-access-5q29v\") pod \"node-ca-bmzq6\" (UID: \"99b873f3-650c-4b66-acf7-5eb80ecb4c70\") " pod="openshift-image-registry/node-ca-bmzq6" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366675 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-system-cni-dir\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366690 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-os-release\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366700 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-multus-cni-dir\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366728 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05fffc12-7ff5-4ae0-9f3d-85267bb7eaef-agent-certs\") pod \"konnectivity-agent-cqtzm\" (UID: \"05fffc12-7ff5-4ae0-9f3d-85267bb7eaef\") " pod="kube-system/konnectivity-agent-cqtzm" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366743 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-system-cni-dir\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366752 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/409fb45f-4128-440a-98ec-58763de6e171-cni-binary-copy\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366802 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/409fb45f-4128-440a-98ec-58763de6e171-multus-cni-dir\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.366993 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/409fb45f-4128-440a-98ec-58763de6e171-multus-daemon-config\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.367072 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/99b873f3-650c-4b66-acf7-5eb80ecb4c70-serviceca\") pod \"node-ca-bmzq6\" (UID: \"99b873f3-650c-4b66-acf7-5eb80ecb4c70\") " pod="openshift-image-registry/node-ca-bmzq6" Jul 01 19:11:39.367742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.367289 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05fffc12-7ff5-4ae0-9f3d-85267bb7eaef-konnectivity-ca\") pod \"konnectivity-agent-cqtzm\" (UID: \"05fffc12-7ff5-4ae0-9f3d-85267bb7eaef\") " pod="kube-system/konnectivity-agent-cqtzm" Jul 01 19:11:39.368290 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.367296 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/409fb45f-4128-440a-98ec-58763de6e171-cni-binary-copy\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.369425 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.369401 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05fffc12-7ff5-4ae0-9f3d-85267bb7eaef-agent-certs\") pod \"konnectivity-agent-cqtzm\" (UID: \"05fffc12-7ff5-4ae0-9f3d-85267bb7eaef\") " pod="kube-system/konnectivity-agent-cqtzm" Jul 01 19:11:39.376320 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.376297 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sxdz\" (UniqueName: \"kubernetes.io/projected/409fb45f-4128-440a-98ec-58763de6e171-kube-api-access-4sxdz\") pod \"multus-24dr2\" (UID: \"409fb45f-4128-440a-98ec-58763de6e171\") " pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.376580 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.376555 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q29v\" (UniqueName: \"kubernetes.io/projected/99b873f3-650c-4b66-acf7-5eb80ecb4c70-kube-api-access-5q29v\") pod \"node-ca-bmzq6\" (UID: \"99b873f3-650c-4b66-acf7-5eb80ecb4c70\") " pod="openshift-image-registry/node-ca-bmzq6" Jul 01 19:11:39.452961 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.452860 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:11:39.462282 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.462254 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" Jul 01 19:11:39.470316 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.470293 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xlfr9" Jul 01 19:11:39.476284 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.476263 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" Jul 01 19:11:39.482174 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.482153 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7tp8j" Jul 01 19:11:39.491218 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.491195 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x2scp" Jul 01 19:11:39.498855 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.498835 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cqtzm" Jul 01 19:11:39.507648 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.507622 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bmzq6" Jul 01 19:11:39.511466 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.511302 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-24dr2" Jul 01 19:11:39.770646 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.770552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:39.770825 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.770734 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:11:39.770825 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.770812 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs podName:47beecfe-aae0-485b-b9f4-9c75b22096ea nodeName:}" failed. No retries permitted until 2026-07-01 19:11:40.770792231 +0000 UTC m=+3.998966766 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs") pod "network-metrics-daemon-cbmfg" (UID: "47beecfe-aae0-485b-b9f4-9c75b22096ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:11:39.871520 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.871486 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbgwq\" (UniqueName: \"kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq\") pod \"network-check-target-pl6zn\" (UID: \"b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83\") " pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:39.871707 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:39.871546 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:39.871707 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.871653 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:11:39.871707 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.871681 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:11:39.871707 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.871707 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:11:39.871875 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.871714 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret podName:943fd8e6-36e9-45d0-a60f-78d295d95d9f nodeName:}" failed. No retries permitted until 2026-07-01 19:11:40.871696333 +0000 UTC m=+4.099870871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret") pod "global-pull-secret-syncer-66xvf" (UID: "943fd8e6-36e9-45d0-a60f-78d295d95d9f") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:11:39.871875 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.871722 2577 projected.go:196] Error preparing data for projected volume kube-api-access-sbgwq for pod openshift-network-diagnostics/network-check-target-pl6zn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:11:39.871875 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:39.871778 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq podName:b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83 nodeName:}" failed. No retries permitted until 2026-07-01 19:11:40.871761261 +0000 UTC m=+4.099935814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sbgwq" (UniqueName: "kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq") pod "network-check-target-pl6zn" (UID: "b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:11:40.194090 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:40.194063 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86f9c330_5a8b_49cc_85fc_b285b09a64cb.slice/crio-7c50142a11842365cd7cc56896556890eac604bcb4155c731b0dd379332c0ede WatchSource:0}: Error finding container 7c50142a11842365cd7cc56896556890eac604bcb4155c731b0dd379332c0ede: Status 404 returned error can't find the container with id 7c50142a11842365cd7cc56896556890eac604bcb4155c731b0dd379332c0ede Jul 01 19:11:40.196011 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:40.195984 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod771dc0a4_073a_4c63_85c5_e405dad55641.slice/crio-ed4656ba043ea8b866a123d51a64e4e12854046c8d6ec6337e914a8a2ef93c82 WatchSource:0}: Error finding container ed4656ba043ea8b866a123d51a64e4e12854046c8d6ec6337e914a8a2ef93c82: Status 404 returned error can't find the container with id ed4656ba043ea8b866a123d51a64e4e12854046c8d6ec6337e914a8a2ef93c82 Jul 01 19:11:40.197755 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:40.197729 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60d04e1f_c2df_40a5_9c13_d1109fbc7e69.slice/crio-9f2e35ca2297bcf8937415d02ac0ed22bc82fcfc94998ef05e5744fa7e175f51 WatchSource:0}: Error finding container 9f2e35ca2297bcf8937415d02ac0ed22bc82fcfc94998ef05e5744fa7e175f51: Status 404 returned error can't find the container with id 9f2e35ca2297bcf8937415d02ac0ed22bc82fcfc94998ef05e5744fa7e175f51 Jul 01 19:11:40.200324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.200289 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 19:06:39 +0000 UTC" deadline="2028-02-28 13:07:55.710775693 +0000 UTC" Jul 01 19:11:40.200324 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.200317 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14561h56m15.510463239s" Jul 01 19:11:40.204586 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:40.204564 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5f8c521_cbd7_4873_8b71_9ad27d4bb3f6.slice/crio-99d96bf5cc0720a87a5dbb3523357eb86567f2e3ab1f619396c4b4a75777d1f4 WatchSource:0}: Error finding container 99d96bf5cc0720a87a5dbb3523357eb86567f2e3ab1f619396c4b4a75777d1f4: Status 404 returned error can't find the container with id 99d96bf5cc0720a87a5dbb3523357eb86567f2e3ab1f619396c4b4a75777d1f4 Jul 01 19:11:40.205781 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:40.205756 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99b873f3_650c_4b66_acf7_5eb80ecb4c70.slice/crio-a4da218af1004442a6c4295b986e09d737771792e2dc94bc1e86ff612105ead9 WatchSource:0}: Error finding container a4da218af1004442a6c4295b986e09d737771792e2dc94bc1e86ff612105ead9: Status 404 returned error can't find the container with id a4da218af1004442a6c4295b986e09d737771792e2dc94bc1e86ff612105ead9 Jul 01 19:11:40.206842 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:40.206807 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05fffc12_7ff5_4ae0_9f3d_85267bb7eaef.slice/crio-8f1aed9124cd80f4be0726637a933847ba136b2b3a71008ded2e8d9e9a6bfa88 WatchSource:0}: Error finding container 8f1aed9124cd80f4be0726637a933847ba136b2b3a71008ded2e8d9e9a6bfa88: Status 404 returned error can't find the container with id 8f1aed9124cd80f4be0726637a933847ba136b2b3a71008ded2e8d9e9a6bfa88 Jul 01 19:11:40.208230 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:40.207948 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod691b723a_898b_4dbc_81ee_6646ef35df0a.slice/crio-5add64f6136df6f29b70dd7729e15a06b3e306bc7c1b6cdec9e8fd9ad9992838 WatchSource:0}: Error finding container 5add64f6136df6f29b70dd7729e15a06b3e306bc7c1b6cdec9e8fd9ad9992838: Status 404 returned error can't find the container with id 5add64f6136df6f29b70dd7729e15a06b3e306bc7c1b6cdec9e8fd9ad9992838 Jul 01 19:11:40.208456 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:40.208422 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod409fb45f_4128_440a_98ec_58763de6e171.slice/crio-168b3e3e3fa5b52a083951758b612d848071142feea6b40bdd5a565a58cf6007 WatchSource:0}: Error finding container 168b3e3e3fa5b52a083951758b612d848071142feea6b40bdd5a565a58cf6007: Status 404 returned error can't find the container with id 168b3e3e3fa5b52a083951758b612d848071142feea6b40bdd5a565a58cf6007 Jul 01 19:11:40.210956 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:11:40.209933 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab8ff4c5_e79a_47a7_836d_569f0a408205.slice/crio-7f339f756898e146d87c446a5cac792d6f78c49654998fe63642986362186434 WatchSource:0}: Error finding container 7f339f756898e146d87c446a5cac792d6f78c49654998fe63642986362186434: Status 404 returned error can't find the container with id 7f339f756898e146d87c446a5cac792d6f78c49654998fe63642986362186434 Jul 01 19:11:40.253969 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.253941 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" event={"ID":"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6","Type":"ContainerStarted","Data":"99d96bf5cc0720a87a5dbb3523357eb86567f2e3ab1f619396c4b4a75777d1f4"} Jul 01 19:11:40.255093 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.255062 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" event={"ID":"60d04e1f-c2df-40a5-9c13-d1109fbc7e69","Type":"ContainerStarted","Data":"9f2e35ca2297bcf8937415d02ac0ed22bc82fcfc94998ef05e5744fa7e175f51"} Jul 01 19:11:40.256156 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.256132 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xlfr9" event={"ID":"771dc0a4-073a-4c63-85c5-e405dad55641","Type":"ContainerStarted","Data":"ed4656ba043ea8b866a123d51a64e4e12854046c8d6ec6337e914a8a2ef93c82"} Jul 01 19:11:40.257270 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.257250 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" event={"ID":"86f9c330-5a8b-49cc-85fc-b285b09a64cb","Type":"ContainerStarted","Data":"7c50142a11842365cd7cc56896556890eac604bcb4155c731b0dd379332c0ede"} Jul 01 19:11:40.258370 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.258352 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x2scp" event={"ID":"ab8ff4c5-e79a-47a7-836d-569f0a408205","Type":"ContainerStarted","Data":"7f339f756898e146d87c446a5cac792d6f78c49654998fe63642986362186434"} Jul 01 19:11:40.259424 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.259403 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7tp8j" event={"ID":"691b723a-898b-4dbc-81ee-6646ef35df0a","Type":"ContainerStarted","Data":"5add64f6136df6f29b70dd7729e15a06b3e306bc7c1b6cdec9e8fd9ad9992838"} Jul 01 19:11:40.260530 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.260509 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-24dr2" event={"ID":"409fb45f-4128-440a-98ec-58763de6e171","Type":"ContainerStarted","Data":"168b3e3e3fa5b52a083951758b612d848071142feea6b40bdd5a565a58cf6007"} Jul 01 19:11:40.261565 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.261545 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bmzq6" event={"ID":"99b873f3-650c-4b66-acf7-5eb80ecb4c70","Type":"ContainerStarted","Data":"a4da218af1004442a6c4295b986e09d737771792e2dc94bc1e86ff612105ead9"} Jul 01 19:11:40.262649 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.262627 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cqtzm" event={"ID":"05fffc12-7ff5-4ae0-9f3d-85267bb7eaef","Type":"ContainerStarted","Data":"8f1aed9124cd80f4be0726637a933847ba136b2b3a71008ded2e8d9e9a6bfa88"} Jul 01 19:11:40.657854 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.657389 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:11:40.780266 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.779587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:40.780266 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:40.779770 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:11:40.780266 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:40.779838 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs podName:47beecfe-aae0-485b-b9f4-9c75b22096ea nodeName:}" failed. No retries permitted until 2026-07-01 19:11:42.779818368 +0000 UTC m=+6.007992910 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs") pod "network-metrics-daemon-cbmfg" (UID: "47beecfe-aae0-485b-b9f4-9c75b22096ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:11:40.881138 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.881102 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:40.881329 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:40.881195 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbgwq\" (UniqueName: \"kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq\") pod \"network-check-target-pl6zn\" (UID: \"b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83\") " pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:40.881329 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:40.881320 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:11:40.881463 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:40.881339 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:11:40.881463 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:40.881361 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:11:40.881463 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:40.881375 2577 projected.go:196] Error preparing data for projected volume kube-api-access-sbgwq for pod openshift-network-diagnostics/network-check-target-pl6zn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:11:40.881463 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:40.881395 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret podName:943fd8e6-36e9-45d0-a60f-78d295d95d9f nodeName:}" failed. No retries permitted until 2026-07-01 19:11:42.88137387 +0000 UTC m=+6.109548409 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret") pod "global-pull-secret-syncer-66xvf" (UID: "943fd8e6-36e9-45d0-a60f-78d295d95d9f") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:11:40.881463 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:40.881425 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq podName:b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83 nodeName:}" failed. No retries permitted until 2026-07-01 19:11:42.881409191 +0000 UTC m=+6.109583746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sbgwq" (UniqueName: "kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq") pod "network-check-target-pl6zn" (UID: "b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:11:41.201387 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:41.201235 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 19:06:39 +0000 UTC" deadline="2028-01-13 08:40:13.338585886 +0000 UTC" Jul 01 19:11:41.201387 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:41.201300 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13453h28m32.137290618s" Jul 01 19:11:41.247393 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:41.247362 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:41.247565 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:41.247517 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:11:41.247971 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:41.247946 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:41.248084 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:41.248059 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:11:41.248940 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:41.248914 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:41.249045 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:41.249026 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:11:41.271953 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:41.271921 2577 generic.go:356] "Generic (PLEG): container finished" podID="8ae41eef750adfd1eb4d5837b9a03cd1" containerID="0b7bfb805abdee7cc9c0b48870c84898c0c57344cd31525fc95753331180a989" exitCode=0 Jul 01 19:11:41.272108 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:41.272014 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" event={"ID":"8ae41eef750adfd1eb4d5837b9a03cd1","Type":"ContainerDied","Data":"0b7bfb805abdee7cc9c0b48870c84898c0c57344cd31525fc95753331180a989"} Jul 01 19:11:41.281546 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:41.281517 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-98.ec2.internal" event={"ID":"8247c4c3a47d291912a129baeb7becc3","Type":"ContainerStarted","Data":"c49684a7c74593ea367aeef30e7dee948ca122aabea7459be5c2e24190ba5a85"} Jul 01 19:11:41.304090 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:41.304035 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-98.ec2.internal" podStartSLOduration=3.304013211 podStartE2EDuration="3.304013211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:11:41.30327378 +0000 UTC m=+4.531448341" watchObservedRunningTime="2026-07-01 19:11:41.304013211 +0000 UTC m=+4.532187771" Jul 01 19:11:42.322432 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:42.322369 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" event={"ID":"8ae41eef750adfd1eb4d5837b9a03cd1","Type":"ContainerStarted","Data":"36e3249885ceb1a56aab33dec93a4b995c6d3b14e98c9d8d0ae958776ba3053b"} Jul 01 19:11:42.340588 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:42.340279 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-98.ec2.internal" podStartSLOduration=4.340257027 podStartE2EDuration="4.340257027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:11:42.339813018 +0000 UTC m=+5.567987577" watchObservedRunningTime="2026-07-01 19:11:42.340257027 +0000 UTC m=+5.568431588" Jul 01 19:11:42.797520 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:42.796887 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:42.797520 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:42.797042 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:11:42.797520 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:42.797107 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs podName:47beecfe-aae0-485b-b9f4-9c75b22096ea nodeName:}" failed. No retries permitted until 2026-07-01 19:11:46.797086917 +0000 UTC m=+10.025261454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs") pod "network-metrics-daemon-cbmfg" (UID: "47beecfe-aae0-485b-b9f4-9c75b22096ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:11:42.897691 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:42.897650 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbgwq\" (UniqueName: \"kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq\") pod \"network-check-target-pl6zn\" (UID: \"b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83\") " pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:42.897855 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:42.897731 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:42.897855 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:42.897850 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:11:42.897984 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:42.897908 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret podName:943fd8e6-36e9-45d0-a60f-78d295d95d9f nodeName:}" failed. No retries permitted until 2026-07-01 19:11:46.897890765 +0000 UTC m=+10.126065300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret") pod "global-pull-secret-syncer-66xvf" (UID: "943fd8e6-36e9-45d0-a60f-78d295d95d9f") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:11:42.898331 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:42.898313 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:11:42.898418 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:42.898336 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:11:42.898418 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:42.898349 2577 projected.go:196] Error preparing data for projected volume kube-api-access-sbgwq for pod openshift-network-diagnostics/network-check-target-pl6zn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:11:42.898418 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:42.898389 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq podName:b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83 nodeName:}" failed. No retries permitted until 2026-07-01 19:11:46.89837651 +0000 UTC m=+10.126551047 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sbgwq" (UniqueName: "kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq") pod "network-check-target-pl6zn" (UID: "b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:11:43.250344 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:43.250276 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:43.250701 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:43.250671 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:11:43.251367 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:43.251350 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:43.251680 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:43.251660 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:11:43.254088 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:43.252198 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:43.254088 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:43.252415 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:11:45.245971 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:45.245876 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:45.245971 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:45.245925 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:45.246485 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:45.245884 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:45.246485 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:45.246095 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:11:45.246584 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:45.246547 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:11:45.246691 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:45.246632 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:11:46.831344 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:46.831263 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:46.831768 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:46.831435 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:11:46.831768 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:46.831515 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs podName:47beecfe-aae0-485b-b9f4-9c75b22096ea nodeName:}" failed. No retries permitted until 2026-07-01 19:11:54.831494503 +0000 UTC m=+18.059669057 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs") pod "network-metrics-daemon-cbmfg" (UID: "47beecfe-aae0-485b-b9f4-9c75b22096ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:11:46.933028 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:46.932092 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbgwq\" (UniqueName: \"kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq\") pod \"network-check-target-pl6zn\" (UID: \"b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83\") " pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:46.933028 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:46.932238 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:46.933028 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:46.932398 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:11:46.933028 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:46.932486 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret podName:943fd8e6-36e9-45d0-a60f-78d295d95d9f nodeName:}" failed. No retries permitted until 2026-07-01 19:11:54.932453383 +0000 UTC m=+18.160627919 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret") pod "global-pull-secret-syncer-66xvf" (UID: "943fd8e6-36e9-45d0-a60f-78d295d95d9f") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:11:46.933028 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:46.932919 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:11:46.933028 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:46.932940 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:11:46.933028 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:46.932952 2577 projected.go:196] Error preparing data for projected volume kube-api-access-sbgwq for pod openshift-network-diagnostics/network-check-target-pl6zn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:11:46.933028 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:46.932993 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq podName:b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83 nodeName:}" failed. No retries permitted until 2026-07-01 19:11:54.932980361 +0000 UTC m=+18.161154908 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sbgwq" (UniqueName: "kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq") pod "network-check-target-pl6zn" (UID: "b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:11:47.251669 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:47.250361 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:47.251669 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:47.250545 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:47.251669 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:47.250654 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:11:47.251669 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:47.251164 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:11:47.251669 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:47.251296 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:47.251669 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:47.251473 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:11:49.245939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:49.245884 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:49.245939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:49.245923 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:49.246591 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:49.245889 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:49.246591 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:49.246046 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:11:49.246591 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:49.246154 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:11:49.246591 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:49.246259 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:11:51.246393 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:51.246351 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:51.246847 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:51.246351 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:51.246847 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:51.246509 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:11:51.246847 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:51.246351 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:51.246847 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:51.246585 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:11:51.246847 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:51.246656 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:11:53.246100 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:53.246067 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:53.246100 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:53.246084 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:53.246636 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:53.246067 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:53.246636 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:53.246230 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:11:53.246636 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:53.246272 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:11:53.246636 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:53.246361 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:11:54.896452 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:54.896417 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:54.896879 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:54.896608 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:11:54.896879 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:54.896677 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs podName:47beecfe-aae0-485b-b9f4-9c75b22096ea nodeName:}" failed. No retries permitted until 2026-07-01 19:12:10.896660397 +0000 UTC m=+34.124834931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs") pod "network-metrics-daemon-cbmfg" (UID: "47beecfe-aae0-485b-b9f4-9c75b22096ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:11:54.997105 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:54.997057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:54.997299 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:54.997157 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbgwq\" (UniqueName: \"kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq\") pod \"network-check-target-pl6zn\" (UID: \"b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83\") " pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:54.997299 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:54.997235 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:11:54.997401 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:54.997326 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret podName:943fd8e6-36e9-45d0-a60f-78d295d95d9f nodeName:}" failed. No retries permitted until 2026-07-01 19:12:10.997305082 +0000 UTC m=+34.225479620 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret") pod "global-pull-secret-syncer-66xvf" (UID: "943fd8e6-36e9-45d0-a60f-78d295d95d9f") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:11:54.997401 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:54.997334 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:11:54.997401 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:54.997350 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:11:54.997401 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:54.997358 2577 projected.go:196] Error preparing data for projected volume kube-api-access-sbgwq for pod openshift-network-diagnostics/network-check-target-pl6zn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:11:54.997401 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:54.997401 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq podName:b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:10.997390179 +0000 UTC m=+34.225564714 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sbgwq" (UniqueName: "kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq") pod "network-check-target-pl6zn" (UID: "b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:11:55.248707 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:55.248630 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:55.248831 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:55.248630 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:55.248831 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:55.248738 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:11:55.248831 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:55.248639 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:55.248831 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:55.248796 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:11:55.248962 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:55.248890 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:11:57.246445 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:57.246399 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:57.246946 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:57.246486 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:57.246946 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:57.246499 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:11:57.246946 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:57.246615 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:11:57.246946 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:57.246672 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:57.246946 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:57.246745 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:11:59.246350 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.246142 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:11:59.246736 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:59.246451 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:11:59.246736 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.246529 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:11:59.246736 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:59.246611 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:11:59.246736 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.246687 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:11:59.246932 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:11:59.246778 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:11:59.355859 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.355828 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7tp8j" event={"ID":"691b723a-898b-4dbc-81ee-6646ef35df0a","Type":"ContainerStarted","Data":"7aa28454d41251c7fcaa5a5a754ead4add86449ad97e0e7bee61097ff36798d1"} Jul 01 19:11:59.357409 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.357379 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-24dr2" event={"ID":"409fb45f-4128-440a-98ec-58763de6e171","Type":"ContainerStarted","Data":"a66ba4a9191ca51e3b0bfe9524d88452332903d2b6c41e1839ed51248be3bdd0"} Jul 01 19:11:59.358765 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.358741 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bmzq6" event={"ID":"99b873f3-650c-4b66-acf7-5eb80ecb4c70","Type":"ContainerStarted","Data":"edb0049e1e2ff94dc3c69cf251d84df30db1dfe71173a0e3ae6b475ead8f7de7"} Jul 01 19:11:59.360279 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.360250 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cqtzm" event={"ID":"05fffc12-7ff5-4ae0-9f3d-85267bb7eaef","Type":"ContainerStarted","Data":"383dd5aa2c506333b2cfa01e4c49d6e096b819cf685fcdff39fcb4044313966d"} Jul 01 19:11:59.361776 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.361759 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmnn_f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6/ovn-acl-logging/0.log" Jul 01 19:11:59.362102 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.362083 2577 generic.go:356] "Generic (PLEG): container finished" podID="f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6" containerID="375c7ebb5f4ea220b861c02cdec2e20191b86670fa0bc1400935247db7fead89" exitCode=1 Jul 01 19:11:59.362190 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.362141 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" event={"ID":"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6","Type":"ContainerDied","Data":"375c7ebb5f4ea220b861c02cdec2e20191b86670fa0bc1400935247db7fead89"} Jul 01 19:11:59.362245 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.362205 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" event={"ID":"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6","Type":"ContainerStarted","Data":"c270840290d3ae1b1375a8c3cfc4847e9b670696802bdd0704b28354d32ddb41"} Jul 01 19:11:59.363563 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.363536 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" event={"ID":"60d04e1f-c2df-40a5-9c13-d1109fbc7e69","Type":"ContainerStarted","Data":"37ca6631562bd4a71e3b50be977472df8b5088d11f668150b6f08e877fb3ef9a"} Jul 01 19:11:59.365032 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.365011 2577 generic.go:356] "Generic (PLEG): container finished" podID="771dc0a4-073a-4c63-85c5-e405dad55641" containerID="7a521387660912719c69771861981a38632b71185855941e6b10decba30069d7" exitCode=0 Jul 01 19:11:59.365105 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.365078 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xlfr9" event={"ID":"771dc0a4-073a-4c63-85c5-e405dad55641","Type":"ContainerDied","Data":"7a521387660912719c69771861981a38632b71185855941e6b10decba30069d7"} Jul 01 19:11:59.366665 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.366538 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" event={"ID":"86f9c330-5a8b-49cc-85fc-b285b09a64cb","Type":"ContainerStarted","Data":"572196b61c1dac98ab0d16169baeffd674d0367d67fce849aa1e2af5fd855b5d"} Jul 01 19:11:59.370885 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.370845 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7tp8j" podStartSLOduration=3.559161645 podStartE2EDuration="22.370831034s" totalImagesPullingTime="18.811669389s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:11:59.370345158 +0000 UTC m=+22.598519714" watchObservedRunningTime="2026-07-01 19:11:59.370831034 +0000 UTC m=+22.599005588" Jul 01 19:11:59.385213 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.385178 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lmm5r" podStartSLOduration=3.544580782 podStartE2EDuration="22.385166552s" totalImagesPullingTime="18.84058577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:11:59.385136836 +0000 UTC m=+22.613311388" watchObservedRunningTime="2026-07-01 19:11:59.385166552 +0000 UTC m=+22.613341106" Jul 01 19:11:59.398382 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.398347 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cqtzm" podStartSLOduration=7.31457581 podStartE2EDuration="22.398334709s" totalImagesPullingTime="15.083758899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:11:59.398148417 +0000 UTC m=+22.626322973" watchObservedRunningTime="2026-07-01 19:11:59.398334709 +0000 UTC m=+22.626509265" Jul 01 19:11:59.426897 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.426852 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bmzq6" podStartSLOduration=3.612962566 podStartE2EDuration="22.426837252s" totalImagesPullingTime="18.813874686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:11:59.411568027 +0000 UTC m=+22.639742582" watchObservedRunningTime="2026-07-01 19:11:59.426837252 +0000 UTC m=+22.655011807" Jul 01 19:11:59.427043 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:11:59.426983 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-24dr2" podStartSLOduration=3.561029649 podStartE2EDuration="22.426971642s" totalImagesPullingTime="18.865941993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:11:59.426603104 +0000 UTC m=+22.654777661" watchObservedRunningTime="2026-07-01 19:11:59.426971642 +0000 UTC m=+22.655146198" Jul 01 19:12:00.370760 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:00.370724 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x2scp" event={"ID":"ab8ff4c5-e79a-47a7-836d-569f0a408205","Type":"ContainerStarted","Data":"60bd9322366d3047c5a9726cf2807d710b163e47a06ce1a02780aeea6966f93d"} Jul 01 19:12:00.373657 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:00.373640 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmnn_f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6/ovn-acl-logging/0.log" Jul 01 19:12:00.374086 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:00.374006 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" event={"ID":"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6","Type":"ContainerStarted","Data":"fc0ad250080bb04e69151a32dc5d822a4722488ef30276598f19cd79ef72e895"} Jul 01 19:12:00.374189 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:00.374092 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" event={"ID":"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6","Type":"ContainerStarted","Data":"5562e1216a1ca2a4e744011985020f252a3b83e25847c2a79d4b39940ec2282e"} Jul 01 19:12:00.374189 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:00.374160 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" event={"ID":"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6","Type":"ContainerStarted","Data":"01c770ad4f4a0c691b4c0249ada692d6340d41da1a4d0304581374368e840d02"} Jul 01 19:12:00.374299 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:00.374215 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" event={"ID":"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6","Type":"ContainerStarted","Data":"b503f73eb66b8c164cf7735f3c8e15241273099fc71a42e22638e171b6e63f21"} Jul 01 19:12:00.393945 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:00.393926 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cqtzm" Jul 01 19:12:00.531766 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:00.531676 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 01 19:12:01.230872 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:01.230703 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T19:12:00.531698691Z","UUID":"a587bce5-c624-4b56-96d4-073ea0c47e48","Handler":null,"Name":"","Endpoint":""} Jul 01 19:12:01.232938 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:01.232890 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 01 19:12:01.232938 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:01.232934 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 01 19:12:01.246232 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:01.246143 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:12:01.246232 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:01.246224 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:12:01.246413 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:01.246180 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:12:01.246413 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:01.246313 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:12:01.246525 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:01.246434 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:12:01.246579 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:01.246522 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:12:01.379084 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:01.379051 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" event={"ID":"86f9c330-5a8b-49cc-85fc-b285b09a64cb","Type":"ContainerStarted","Data":"77e0e6b4325c85e372b1e7dcd4026f96fa83a33107207e3db1812fca052895f9"} Jul 01 19:12:01.393851 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:01.393796 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x2scp" podStartSLOduration=5.584254027 podStartE2EDuration="24.393774527s" totalImagesPullingTime="18.8095205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:12:01.393379961 +0000 UTC m=+24.621554519" watchObservedRunningTime="2026-07-01 19:12:01.393774527 +0000 UTC m=+24.621949083" Jul 01 19:12:02.288976 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:02.288934 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cqtzm" Jul 01 19:12:02.289554 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:02.289529 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cqtzm" Jul 01 19:12:02.384789 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:02.384752 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmnn_f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6/ovn-acl-logging/0.log" Jul 01 19:12:02.385313 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:02.385219 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" event={"ID":"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6","Type":"ContainerStarted","Data":"1702817cb525d91430383dffe3df55a4dc41f5496527a605f3419063dbd317ce"} Jul 01 19:12:02.387439 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:02.387416 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" event={"ID":"86f9c330-5a8b-49cc-85fc-b285b09a64cb","Type":"ContainerStarted","Data":"128ff170fcc0b2459566e0236d3073e00c40ce634bbbb5fc455a165274770fc0"} Jul 01 19:12:02.388173 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:02.388156 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cqtzm" Jul 01 19:12:02.404239 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:02.404180 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqqpg" podStartSLOduration=4.288104589 podStartE2EDuration="25.404158065s" totalImagesPullingTime="21.116053476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:11:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:12:02.403616039 +0000 UTC m=+25.631790594" watchObservedRunningTime="2026-07-01 19:12:02.404158065 +0000 UTC m=+25.632332621" Jul 01 19:12:03.246647 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:03.246612 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:12:03.246825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:03.246612 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:12:03.246825 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:03.246748 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:12:03.246825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:03.246612 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:12:03.246825 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:03.246806 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:12:03.247027 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:03.246875 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:12:04.397731 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:04.397527 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmnn_f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6/ovn-acl-logging/0.log" Jul 01 19:12:04.398532 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:04.398010 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" event={"ID":"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6","Type":"ContainerStarted","Data":"0eb6373956bb83cbce743a3cac69bcce270ec9d5f020a0eb290a6c39f6c7285d"} Jul 01 19:12:04.398532 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:04.398320 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:12:04.398532 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:04.398404 2577 scope.go:117] "RemoveContainer" containerID="375c7ebb5f4ea220b861c02cdec2e20191b86670fa0bc1400935247db7fead89" Jul 01 19:12:04.399923 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:04.399899 2577 generic.go:356] "Generic (PLEG): container finished" podID="771dc0a4-073a-4c63-85c5-e405dad55641" containerID="ddd85023887a0f8b9cb5b42be3fd95a4701dafb55395b123733fa7db1d101017" exitCode=0 Jul 01 19:12:04.400053 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:04.399960 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xlfr9" event={"ID":"771dc0a4-073a-4c63-85c5-e405dad55641","Type":"ContainerDied","Data":"ddd85023887a0f8b9cb5b42be3fd95a4701dafb55395b123733fa7db1d101017"} Jul 01 19:12:04.416596 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:04.416574 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:12:05.246052 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:05.245807 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:12:05.246256 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:05.245815 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:12:05.246256 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:05.246082 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:12:05.246256 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:05.245815 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:12:05.246256 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:05.246198 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:12:05.246256 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:05.246241 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:12:05.405753 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:05.405726 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmnn_f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6/ovn-acl-logging/0.log" Jul 01 19:12:05.406156 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:05.406057 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" event={"ID":"f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6","Type":"ContainerStarted","Data":"8a4dd537b68a3911725a36b63a82a24156bbfd8d6eb62bcae67092475669bdce"} Jul 01 19:12:05.406156 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:05.406149 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 01 19:12:05.406412 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:05.406387 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:12:05.421444 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:05.421423 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:12:05.432605 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:05.432564 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" podStartSLOduration=9.519162912 podStartE2EDuration="28.432548295s" totalImagesPullingTime="18.913385383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:12:05.431828644 +0000 UTC m=+28.660003198" watchObservedRunningTime="2026-07-01 19:12:05.432548295 +0000 UTC m=+28.660722850" Jul 01 19:12:06.113306 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:06.113275 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-66xvf"] Jul 01 19:12:06.113530 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:06.113434 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:12:06.113600 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:06.113545 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:12:06.115072 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:06.115044 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cbmfg"] Jul 01 19:12:06.115200 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:06.115176 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:12:06.115308 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:06.115277 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:12:06.125770 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:06.125747 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pl6zn"] Jul 01 19:12:06.125869 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:06.125829 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:12:06.125908 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:06.125890 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:12:06.411486 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:06.411456 2577 generic.go:356] "Generic (PLEG): container finished" podID="771dc0a4-073a-4c63-85c5-e405dad55641" containerID="2da6697b7eb0191fa1ba13034125b87039cadfe3a9f98b754b11d2bf81f5e699" exitCode=0 Jul 01 19:12:06.412082 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:06.411532 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xlfr9" event={"ID":"771dc0a4-073a-4c63-85c5-e405dad55641","Type":"ContainerDied","Data":"2da6697b7eb0191fa1ba13034125b87039cadfe3a9f98b754b11d2bf81f5e699"} Jul 01 19:12:06.412082 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:06.411625 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 01 19:12:07.247354 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:07.247329 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:12:07.247507 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:07.247407 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:12:07.247507 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:07.247450 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:12:07.247577 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:07.247504 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:12:07.413920 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:07.413890 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 01 19:12:07.493667 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:07.493634 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:12:08.246076 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:08.246041 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:12:08.246285 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:08.246180 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:12:08.419089 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:08.419010 2577 generic.go:356] "Generic (PLEG): container finished" podID="771dc0a4-073a-4c63-85c5-e405dad55641" containerID="6719dfbfbaf171ec3f1c08f901e93a23c5bed17f8d6e0ce4b52001508d176f47" exitCode=0 Jul 01 19:12:08.419089 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:08.419076 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xlfr9" event={"ID":"771dc0a4-073a-4c63-85c5-e405dad55641","Type":"ContainerDied","Data":"6719dfbfbaf171ec3f1c08f901e93a23c5bed17f8d6e0ce4b52001508d176f47"} Jul 01 19:12:09.246145 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:09.245887 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:12:09.246314 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:09.245935 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:12:09.246314 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:09.246280 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:12:09.246431 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:09.246320 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:12:10.245687 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:10.245656 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:12:10.246155 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:10.245773 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pl6zn" podUID="b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83" Jul 01 19:12:10.918569 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:10.918536 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:12:10.918729 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:10.918694 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:12:10.918766 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:10.918759 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs podName:47beecfe-aae0-485b-b9f4-9c75b22096ea nodeName:}" failed. No retries permitted until 2026-07-01 19:12:42.918744105 +0000 UTC m=+66.146918638 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs") pod "network-metrics-daemon-cbmfg" (UID: "47beecfe-aae0-485b-b9f4-9c75b22096ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:12:11.019394 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:11.019365 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:12:11.019565 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:11.019435 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbgwq\" (UniqueName: \"kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq\") pod \"network-check-target-pl6zn\" (UID: \"b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83\") " pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:12:11.019565 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:11.019561 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:12:11.019665 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:11.019576 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:12:11.019665 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:11.019577 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:12:11.019665 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:11.019660 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret podName:943fd8e6-36e9-45d0-a60f-78d295d95d9f nodeName:}" failed. No retries permitted until 2026-07-01 19:12:43.01962735 +0000 UTC m=+66.247801902 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret") pod "global-pull-secret-syncer-66xvf" (UID: "943fd8e6-36e9-45d0-a60f-78d295d95d9f") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:12:11.019797 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:11.019586 2577 projected.go:196] Error preparing data for projected volume kube-api-access-sbgwq for pod openshift-network-diagnostics/network-check-target-pl6zn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:12:11.019797 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:11.019731 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq podName:b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:43.019718622 +0000 UTC m=+66.247893154 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sbgwq" (UniqueName: "kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq") pod "network-check-target-pl6zn" (UID: "b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:12:11.245839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:11.245801 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:12:11.246326 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:11.245800 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:12:11.246326 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:11.245934 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbmfg" podUID="47beecfe-aae0-485b-b9f4-9c75b22096ea" Jul 01 19:12:11.246326 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:11.246029 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66xvf" podUID="943fd8e6-36e9-45d0-a60f-78d295d95d9f" Jul 01 19:12:12.021032 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.021005 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-98.ec2.internal" event="NodeReady" Jul 01 19:12:12.021221 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.021174 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 01 19:12:12.061808 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.061735 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rcb2k"] Jul 01 19:12:12.091777 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.091745 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6q7sh"] Jul 01 19:12:12.091955 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.091890 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.094157 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.094071 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 01 19:12:12.094157 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.094093 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6ddhd\"" Jul 01 19:12:12.094157 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.094074 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 01 19:12:12.107689 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.107668 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rcb2k"] Jul 01 19:12:12.107809 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.107712 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6q7sh"] Jul 01 19:12:12.107809 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.107786 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6q7sh" Jul 01 19:12:12.110276 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.110224 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 01 19:12:12.110276 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.110259 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 01 19:12:12.110276 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.110263 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-czspz\"" Jul 01 19:12:12.110632 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.110616 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 01 19:12:12.228859 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.228820 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8855e158-eba7-4470-b0be-cce54da3d10b-tmp-dir\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.229047 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.228867 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2sss\" (UniqueName: \"kubernetes.io/projected/8855e158-eba7-4470-b0be-cce54da3d10b-kube-api-access-z2sss\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.229047 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.228995 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ab889f3-cd15-4323-b69a-e8b1141710ec-cert\") pod \"ingress-canary-6q7sh\" (UID: \"9ab889f3-cd15-4323-b69a-e8b1141710ec\") " pod="openshift-ingress-canary/ingress-canary-6q7sh" Jul 01 19:12:12.229047 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.229046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.229215 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.229072 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcxwf\" (UniqueName: \"kubernetes.io/projected/9ab889f3-cd15-4323-b69a-e8b1141710ec-kube-api-access-pcxwf\") pod \"ingress-canary-6q7sh\" (UID: \"9ab889f3-cd15-4323-b69a-e8b1141710ec\") " pod="openshift-ingress-canary/ingress-canary-6q7sh" Jul 01 19:12:12.229215 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.229148 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8855e158-eba7-4470-b0be-cce54da3d10b-config-volume\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.246682 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.246652 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:12:12.249228 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.249149 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 19:12:12.249228 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.249149 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 19:12:12.249228 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.249200 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-88jrr\"" Jul 01 19:12:12.330356 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.330268 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8855e158-eba7-4470-b0be-cce54da3d10b-tmp-dir\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.330356 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.330321 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2sss\" (UniqueName: \"kubernetes.io/projected/8855e158-eba7-4470-b0be-cce54da3d10b-kube-api-access-z2sss\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.330578 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.330396 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ab889f3-cd15-4323-b69a-e8b1141710ec-cert\") pod \"ingress-canary-6q7sh\" (UID: \"9ab889f3-cd15-4323-b69a-e8b1141710ec\") " pod="openshift-ingress-canary/ingress-canary-6q7sh" Jul 01 19:12:12.330578 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.330418 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.330578 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.330452 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcxwf\" (UniqueName: \"kubernetes.io/projected/9ab889f3-cd15-4323-b69a-e8b1141710ec-kube-api-access-pcxwf\") pod \"ingress-canary-6q7sh\" (UID: \"9ab889f3-cd15-4323-b69a-e8b1141710ec\") " pod="openshift-ingress-canary/ingress-canary-6q7sh" Jul 01 19:12:12.330578 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.330474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8855e158-eba7-4470-b0be-cce54da3d10b-config-volume\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.330578 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:12.330538 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 19:12:12.330809 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:12.330614 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls podName:8855e158-eba7-4470-b0be-cce54da3d10b nodeName:}" failed. No retries permitted until 2026-07-01 19:12:12.830593456 +0000 UTC m=+36.058767989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls") pod "dns-default-rcb2k" (UID: "8855e158-eba7-4470-b0be-cce54da3d10b") : secret "dns-default-metrics-tls" not found Jul 01 19:12:12.330809 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.330700 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8855e158-eba7-4470-b0be-cce54da3d10b-tmp-dir\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.330996 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.330975 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8855e158-eba7-4470-b0be-cce54da3d10b-config-volume\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.335476 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.335452 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ab889f3-cd15-4323-b69a-e8b1141710ec-cert\") pod \"ingress-canary-6q7sh\" (UID: \"9ab889f3-cd15-4323-b69a-e8b1141710ec\") " pod="openshift-ingress-canary/ingress-canary-6q7sh" Jul 01 19:12:12.342531 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.342503 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2sss\" (UniqueName: \"kubernetes.io/projected/8855e158-eba7-4470-b0be-cce54da3d10b-kube-api-access-z2sss\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.342818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.342798 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcxwf\" (UniqueName: \"kubernetes.io/projected/9ab889f3-cd15-4323-b69a-e8b1141710ec-kube-api-access-pcxwf\") pod \"ingress-canary-6q7sh\" (UID: \"9ab889f3-cd15-4323-b69a-e8b1141710ec\") " pod="openshift-ingress-canary/ingress-canary-6q7sh" Jul 01 19:12:12.417855 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.417824 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6q7sh" Jul 01 19:12:12.621867 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.621804 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6q7sh"] Jul 01 19:12:12.662489 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:12:12.662457 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab889f3_cd15_4323_b69a_e8b1141710ec.slice/crio-a0422187c4cb1e9e6667f4b565f0f4d39284b3f2992ea59015324f4d7a0d4ace WatchSource:0}: Error finding container a0422187c4cb1e9e6667f4b565f0f4d39284b3f2992ea59015324f4d7a0d4ace: Status 404 returned error can't find the container with id a0422187c4cb1e9e6667f4b565f0f4d39284b3f2992ea59015324f4d7a0d4ace Jul 01 19:12:12.834820 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:12.834633 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:12.834943 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:12.834778 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 19:12:12.834943 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:12.834925 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls podName:8855e158-eba7-4470-b0be-cce54da3d10b nodeName:}" failed. No retries permitted until 2026-07-01 19:12:13.83491128 +0000 UTC m=+37.063085813 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls") pod "dns-default-rcb2k" (UID: "8855e158-eba7-4470-b0be-cce54da3d10b") : secret "dns-default-metrics-tls" not found Jul 01 19:12:13.245899 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:13.245866 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:12:13.245899 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:13.245879 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:12:13.248679 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:13.248654 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 19:12:13.249501 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:13.249360 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 19:12:13.249607 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:13.249586 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9wqsv\"" Jul 01 19:12:13.434030 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:13.433999 2577 generic.go:356] "Generic (PLEG): container finished" podID="771dc0a4-073a-4c63-85c5-e405dad55641" containerID="8e1af2e1fc64ffa5d920c39feac653ecde29ce0b774726325eb33d60732687fe" exitCode=0 Jul 01 19:12:13.434223 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:13.434089 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xlfr9" event={"ID":"771dc0a4-073a-4c63-85c5-e405dad55641","Type":"ContainerDied","Data":"8e1af2e1fc64ffa5d920c39feac653ecde29ce0b774726325eb33d60732687fe"} Jul 01 19:12:13.435329 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:13.435307 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6q7sh" event={"ID":"9ab889f3-cd15-4323-b69a-e8b1141710ec","Type":"ContainerStarted","Data":"a0422187c4cb1e9e6667f4b565f0f4d39284b3f2992ea59015324f4d7a0d4ace"} Jul 01 19:12:13.841782 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:13.841701 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:13.841944 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:13.841866 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 19:12:13.841999 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:13.841946 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls podName:8855e158-eba7-4470-b0be-cce54da3d10b nodeName:}" failed. No retries permitted until 2026-07-01 19:12:15.841925465 +0000 UTC m=+39.070100004 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls") pod "dns-default-rcb2k" (UID: "8855e158-eba7-4470-b0be-cce54da3d10b") : secret "dns-default-metrics-tls" not found Jul 01 19:12:14.441536 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:14.441495 2577 generic.go:356] "Generic (PLEG): container finished" podID="771dc0a4-073a-4c63-85c5-e405dad55641" containerID="0fd0a6262b7081d66b6a6fa06a33dc94ead6d39719b548ddb683c0d1e9c137ee" exitCode=0 Jul 01 19:12:14.441953 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:14.441551 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xlfr9" event={"ID":"771dc0a4-073a-4c63-85c5-e405dad55641","Type":"ContainerDied","Data":"0fd0a6262b7081d66b6a6fa06a33dc94ead6d39719b548ddb683c0d1e9c137ee"} Jul 01 19:12:15.446782 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:15.446745 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xlfr9" event={"ID":"771dc0a4-073a-4c63-85c5-e405dad55641","Type":"ContainerStarted","Data":"1afd2e796b42dfdba2375436ba314c231112f2b39811256f000da35cc229c38b"} Jul 01 19:12:15.448225 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:15.448203 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6q7sh" event={"ID":"9ab889f3-cd15-4323-b69a-e8b1141710ec","Type":"ContainerStarted","Data":"7963e23f050c89cb818764f407e6e9477ec019d273eba02f9339497de81b444a"} Jul 01 19:12:15.468662 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:15.468589 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xlfr9" podStartSLOduration=8.592771564 podStartE2EDuration="38.468573769s" totalImagesPullingTime="29.875802205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:11:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:12:15.467625217 +0000 UTC m=+38.695799784" watchObservedRunningTime="2026-07-01 19:12:15.468573769 +0000 UTC m=+38.696748326" Jul 01 19:12:15.483298 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:15.483249 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6q7sh" podStartSLOduration=1.172424239 podStartE2EDuration="3.483232898s" totalImagesPullingTime="2.310808659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:12:15.482898589 +0000 UTC m=+38.711073147" watchObservedRunningTime="2026-07-01 19:12:15.483232898 +0000 UTC m=+38.711407454" Jul 01 19:12:15.856761 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:15.856706 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:15.856962 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:15.856843 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 19:12:15.856962 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:15.856920 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls podName:8855e158-eba7-4470-b0be-cce54da3d10b nodeName:}" failed. No retries permitted until 2026-07-01 19:12:19.856902517 +0000 UTC m=+43.085077050 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls") pod "dns-default-rcb2k" (UID: "8855e158-eba7-4470-b0be-cce54da3d10b") : secret "dns-default-metrics-tls" not found Jul 01 19:12:19.883765 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:19.883725 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:19.884157 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:19.883838 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 19:12:19.884157 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:19.883885 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls podName:8855e158-eba7-4470-b0be-cce54da3d10b nodeName:}" failed. No retries permitted until 2026-07-01 19:12:27.883872946 +0000 UTC m=+51.112047480 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls") pod "dns-default-rcb2k" (UID: "8855e158-eba7-4470-b0be-cce54da3d10b") : secret "dns-default-metrics-tls" not found Jul 01 19:12:27.937970 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:27.937926 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:27.938448 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:27.938069 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 19:12:27.938448 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:27.938152 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls podName:8855e158-eba7-4470-b0be-cce54da3d10b nodeName:}" failed. No retries permitted until 2026-07-01 19:12:43.938131113 +0000 UTC m=+67.166305660 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls") pod "dns-default-rcb2k" (UID: "8855e158-eba7-4470-b0be-cce54da3d10b") : secret "dns-default-metrics-tls" not found Jul 01 19:12:32.140431 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.140400 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s"] Jul 01 19:12:32.160512 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.160489 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s"] Jul 01 19:12:32.160649 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.160571 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:32.162798 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.162773 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 01 19:12:32.162910 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.162773 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 01 19:12:32.163553 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.163534 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4sdhh\"" Jul 01 19:12:32.163973 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.163958 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 01 19:12:32.164058 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.163990 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 01 19:12:32.235389 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.235360 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-k8mbf"] Jul 01 19:12:32.251644 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.251603 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7b759854c7-dwz6b"] Jul 01 19:12:32.251800 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.251724 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-k8mbf" Jul 01 19:12:32.253809 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.253786 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 01 19:12:32.253972 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.253841 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 01 19:12:32.254043 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.253846 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9g5r7\"" Jul 01 19:12:32.268611 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.268588 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2rq8\" (UniqueName: \"kubernetes.io/projected/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-kube-api-access-v2rq8\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:32.268714 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.268619 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:32.268714 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.268640 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:32.276586 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.276566 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-k8mbf"] Jul 01 19:12:32.276649 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.276595 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b759854c7-dwz6b"] Jul 01 19:12:32.276685 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.276670 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.278622 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.278590 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 01 19:12:32.278622 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.278618 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 01 19:12:32.278758 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.278626 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 01 19:12:32.278928 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.278911 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 01 19:12:32.279188 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.279173 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 01 19:12:32.279236 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.279210 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 01 19:12:32.279479 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.279465 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9v92p\"" Jul 01 19:12:32.369570 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.369539 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr8ct\" (UniqueName: \"kubernetes.io/projected/9cf3071f-e57f-4f32-9ee8-5f441663ad3c-kube-api-access-vr8ct\") pod \"volume-data-source-validator-69d9f679db-k8mbf\" (UID: \"9cf3071f-e57f-4f32-9ee8-5f441663ad3c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-k8mbf" Jul 01 19:12:32.369717 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.369575 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.369717 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.369606 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:32.369717 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.369674 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:32.369818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.369734 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.369818 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:32.369765 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 19:12:32.369818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.369779 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-default-certificate\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.369818 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.369796 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-stats-auth\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.369989 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.369832 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q96sk\" (UniqueName: \"kubernetes.io/projected/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-kube-api-access-q96sk\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.369989 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:32.369851 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls podName:6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:32.869830204 +0000 UTC m=+56.098004738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-vxf6s" (UID: "6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0") : secret "cluster-monitoring-operator-tls" not found Jul 01 19:12:32.370081 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.369981 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2rq8\" (UniqueName: \"kubernetes.io/projected/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-kube-api-access-v2rq8\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:32.370263 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.370245 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:32.381246 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.381218 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2rq8\" (UniqueName: \"kubernetes.io/projected/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-kube-api-access-v2rq8\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:32.471213 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.471179 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr8ct\" (UniqueName: \"kubernetes.io/projected/9cf3071f-e57f-4f32-9ee8-5f441663ad3c-kube-api-access-vr8ct\") pod \"volume-data-source-validator-69d9f679db-k8mbf\" (UID: \"9cf3071f-e57f-4f32-9ee8-5f441663ad3c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-k8mbf" Jul 01 19:12:32.471397 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.471216 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.471397 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.471265 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.471397 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.471306 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-default-certificate\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.471397 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.471328 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-stats-auth\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.471587 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:32.471408 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle podName:d0c6e2e7-7f8a-4911-a5e1-1933759bb182 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:32.971391497 +0000 UTC m=+56.199566066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle") pod "router-default-7b759854c7-dwz6b" (UID: "d0c6e2e7-7f8a-4911-a5e1-1933759bb182") : configmap references non-existent config key: service-ca.crt Jul 01 19:12:32.471587 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:32.471417 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 19:12:32.471587 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.471443 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q96sk\" (UniqueName: \"kubernetes.io/projected/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-kube-api-access-q96sk\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.471587 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:32.471458 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs podName:d0c6e2e7-7f8a-4911-a5e1-1933759bb182 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:32.971448522 +0000 UTC m=+56.199623055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs") pod "router-default-7b759854c7-dwz6b" (UID: "d0c6e2e7-7f8a-4911-a5e1-1933759bb182") : secret "router-metrics-certs-default" not found Jul 01 19:12:32.473793 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.473768 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-default-certificate\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.473896 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.473773 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-stats-auth\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.479525 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.479503 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr8ct\" (UniqueName: \"kubernetes.io/projected/9cf3071f-e57f-4f32-9ee8-5f441663ad3c-kube-api-access-vr8ct\") pod \"volume-data-source-validator-69d9f679db-k8mbf\" (UID: \"9cf3071f-e57f-4f32-9ee8-5f441663ad3c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-k8mbf" Jul 01 19:12:32.479634 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.479566 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q96sk\" (UniqueName: \"kubernetes.io/projected/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-kube-api-access-q96sk\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.560790 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.560756 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-k8mbf" Jul 01 19:12:32.705658 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.705628 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-k8mbf"] Jul 01 19:12:32.709615 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:12:32.709590 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf3071f_e57f_4f32_9ee8_5f441663ad3c.slice/crio-7f7f4d1aebf8b3680f3139bf9413e13b6726cd0d3a9f5b674cb2b9801b29e463 WatchSource:0}: Error finding container 7f7f4d1aebf8b3680f3139bf9413e13b6726cd0d3a9f5b674cb2b9801b29e463: Status 404 returned error can't find the container with id 7f7f4d1aebf8b3680f3139bf9413e13b6726cd0d3a9f5b674cb2b9801b29e463 Jul 01 19:12:32.874354 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.874282 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:32.874491 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:32.874394 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 19:12:32.874531 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:32.874508 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls podName:6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:33.874433511 +0000 UTC m=+57.102608045 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-vxf6s" (UID: "6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0") : secret "cluster-monitoring-operator-tls" not found Jul 01 19:12:32.975264 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.975234 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.975416 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:32.975304 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:32.975416 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:32.975407 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle podName:d0c6e2e7-7f8a-4911-a5e1-1933759bb182 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:33.975391475 +0000 UTC m=+57.203566023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle") pod "router-default-7b759854c7-dwz6b" (UID: "d0c6e2e7-7f8a-4911-a5e1-1933759bb182") : configmap references non-existent config key: service-ca.crt Jul 01 19:12:32.975498 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:32.975407 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 19:12:32.975498 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:32.975438 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs podName:d0c6e2e7-7f8a-4911-a5e1-1933759bb182 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:33.97543248 +0000 UTC m=+57.203607017 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs") pod "router-default-7b759854c7-dwz6b" (UID: "d0c6e2e7-7f8a-4911-a5e1-1933759bb182") : secret "router-metrics-certs-default" not found Jul 01 19:12:33.491837 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:33.491789 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-k8mbf" event={"ID":"9cf3071f-e57f-4f32-9ee8-5f441663ad3c","Type":"ContainerStarted","Data":"7f7f4d1aebf8b3680f3139bf9413e13b6726cd0d3a9f5b674cb2b9801b29e463"} Jul 01 19:12:33.883884 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:33.883800 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:33.884050 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:33.883993 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 19:12:33.884100 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:33.884086 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls podName:6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:35.884062146 +0000 UTC m=+59.112236694 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-vxf6s" (UID: "6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0") : secret "cluster-monitoring-operator-tls" not found Jul 01 19:12:33.984349 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:33.984312 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:33.984521 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:33.984424 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:33.984521 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:33.984492 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 19:12:33.984625 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:33.984566 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle podName:d0c6e2e7-7f8a-4911-a5e1-1933759bb182 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:35.984547195 +0000 UTC m=+59.212721731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle") pod "router-default-7b759854c7-dwz6b" (UID: "d0c6e2e7-7f8a-4911-a5e1-1933759bb182") : configmap references non-existent config key: service-ca.crt Jul 01 19:12:33.984625 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:33.984596 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs podName:d0c6e2e7-7f8a-4911-a5e1-1933759bb182 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:35.98457656 +0000 UTC m=+59.212751107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs") pod "router-default-7b759854c7-dwz6b" (UID: "d0c6e2e7-7f8a-4911-a5e1-1933759bb182") : secret "router-metrics-certs-default" not found Jul 01 19:12:34.495613 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:34.495588 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-k8mbf" event={"ID":"9cf3071f-e57f-4f32-9ee8-5f441663ad3c","Type":"ContainerStarted","Data":"ee8f7f391f0c8f22e608198ec1cce4a626034b95315bffe5c2d0fb94b4f2c1a2"} Jul 01 19:12:35.511939 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:35.511888 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-k8mbf" podStartSLOduration=1.836583514 podStartE2EDuration="3.511874094s" totalImagesPullingTime="1.67529058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:12:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:12:35.511414254 +0000 UTC m=+58.739588824" watchObservedRunningTime="2026-07-01 19:12:35.511874094 +0000 UTC m=+58.740048649" Jul 01 19:12:35.900182 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:35.900095 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:35.900316 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:35.900264 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 19:12:35.900361 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:35.900339 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls podName:6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:39.900323994 +0000 UTC m=+63.128498527 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-vxf6s" (UID: "6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0") : secret "cluster-monitoring-operator-tls" not found Jul 01 19:12:36.000839 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.000806 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:36.000996 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.000875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:36.000996 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:36.000969 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 19:12:36.001064 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:36.000977 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle podName:d0c6e2e7-7f8a-4911-a5e1-1933759bb182 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:40.00095983 +0000 UTC m=+63.229134389 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle") pod "router-default-7b759854c7-dwz6b" (UID: "d0c6e2e7-7f8a-4911-a5e1-1933759bb182") : configmap references non-existent config key: service-ca.crt Jul 01 19:12:36.001064 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:36.001028 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs podName:d0c6e2e7-7f8a-4911-a5e1-1933759bb182 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:40.001015596 +0000 UTC m=+63.229190149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs") pod "router-default-7b759854c7-dwz6b" (UID: "d0c6e2e7-7f8a-4911-a5e1-1933759bb182") : secret "router-metrics-certs-default" not found Jul 01 19:12:36.184622 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.184547 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rtwv8"] Jul 01 19:12:36.187254 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.187236 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.188971 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.188949 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 01 19:12:36.189077 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.189037 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 01 19:12:36.189225 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.189210 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-54dhx\"" Jul 01 19:12:36.189310 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.189293 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 01 19:12:36.189566 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.189553 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 01 19:12:36.203413 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.203391 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rtwv8"] Jul 01 19:12:36.302832 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.302799 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.302976 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.302844 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/19a0267f-762f-4fb6-971d-f5988a038b5a-crio-socket\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.302976 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.302917 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.302976 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.302943 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/19a0267f-762f-4fb6-971d-f5988a038b5a-data-volume\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.302976 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.302960 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99hlw\" (UniqueName: \"kubernetes.io/projected/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-api-access-99hlw\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.403458 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.403410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.403458 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.403459 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/19a0267f-762f-4fb6-971d-f5988a038b5a-data-volume\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.403699 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.403479 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99hlw\" (UniqueName: \"kubernetes.io/projected/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-api-access-99hlw\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.403699 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.403531 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.403699 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.403558 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/19a0267f-762f-4fb6-971d-f5988a038b5a-crio-socket\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.403699 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:36.403530 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 19:12:36.403699 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.403646 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/19a0267f-762f-4fb6-971d-f5988a038b5a-crio-socket\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.403699 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:36.403653 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 19:12:36.403699 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:36.403665 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm podName:19a0267f-762f-4fb6-971d-f5988a038b5a nodeName:}" failed. No retries permitted until 2026-07-01 19:12:36.903643225 +0000 UTC m=+60.131817773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rtwv8" (UID: "19a0267f-762f-4fb6-971d-f5988a038b5a") : configmap "kube-rbac-proxy" not found Jul 01 19:12:36.404027 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:36.403706 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls podName:19a0267f-762f-4fb6-971d-f5988a038b5a nodeName:}" failed. No retries permitted until 2026-07-01 19:12:36.903691974 +0000 UTC m=+60.131866507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rtwv8" (UID: "19a0267f-762f-4fb6-971d-f5988a038b5a") : secret "insights-runtime-extractor-tls" not found Jul 01 19:12:36.404027 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.403827 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/19a0267f-762f-4fb6-971d-f5988a038b5a-data-volume\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.411937 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.411902 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99hlw\" (UniqueName: \"kubernetes.io/projected/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-api-access-99hlw\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.908326 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.908271 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.908779 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:36.908426 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 19:12:36.908779 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:36.908450 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:36.908779 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:36.908493 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm podName:19a0267f-762f-4fb6-971d-f5988a038b5a nodeName:}" failed. No retries permitted until 2026-07-01 19:12:37.908478012 +0000 UTC m=+61.136652545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rtwv8" (UID: "19a0267f-762f-4fb6-971d-f5988a038b5a") : configmap "kube-rbac-proxy" not found Jul 01 19:12:36.908779 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:36.908538 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 19:12:36.908779 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:36.908580 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls podName:19a0267f-762f-4fb6-971d-f5988a038b5a nodeName:}" failed. No retries permitted until 2026-07-01 19:12:37.908569607 +0000 UTC m=+61.136744139 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rtwv8" (UID: "19a0267f-762f-4fb6-971d-f5988a038b5a") : secret "insights-runtime-extractor-tls" not found Jul 01 19:12:37.916439 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:37.916393 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:37.916797 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:37.916472 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:37.916797 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:37.916519 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 19:12:37.916797 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:37.916585 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm podName:19a0267f-762f-4fb6-971d-f5988a038b5a nodeName:}" failed. No retries permitted until 2026-07-01 19:12:39.916571293 +0000 UTC m=+63.144745830 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rtwv8" (UID: "19a0267f-762f-4fb6-971d-f5988a038b5a") : configmap "kube-rbac-proxy" not found Jul 01 19:12:37.916797 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:37.916617 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 19:12:37.916797 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:37.916669 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls podName:19a0267f-762f-4fb6-971d-f5988a038b5a nodeName:}" failed. No retries permitted until 2026-07-01 19:12:39.916656223 +0000 UTC m=+63.144830756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rtwv8" (UID: "19a0267f-762f-4fb6-971d-f5988a038b5a") : secret "insights-runtime-extractor-tls" not found Jul 01 19:12:37.969627 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:37.969604 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7tp8j_691b723a-898b-4dbc-81ee-6646ef35df0a/dns-node-resolver/0.log" Jul 01 19:12:38.429091 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:38.429066 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksmnn" Jul 01 19:12:38.969092 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:38.969063 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bmzq6_99b873f3-650c-4b66-acf7-5eb80ecb4c70/node-ca/0.log" Jul 01 19:12:39.569470 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:39.569437 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6q7sh_9ab889f3-cd15-4323-b69a-e8b1141710ec/serve-healthcheck-canary/0.log" Jul 01 19:12:39.928841 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:39.928762 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:39.928841 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:39.928821 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:39.928841 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:39.928843 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:39.929039 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:39.928917 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 19:12:39.929039 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:39.928937 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 19:12:39.929039 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:39.928979 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls podName:19a0267f-762f-4fb6-971d-f5988a038b5a nodeName:}" failed. No retries permitted until 2026-07-01 19:12:43.928962688 +0000 UTC m=+67.157137220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rtwv8" (UID: "19a0267f-762f-4fb6-971d-f5988a038b5a") : secret "insights-runtime-extractor-tls" not found Jul 01 19:12:39.929039 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:39.928995 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls podName:6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:47.92898814 +0000 UTC m=+71.157162673 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-vxf6s" (UID: "6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0") : secret "cluster-monitoring-operator-tls" not found Jul 01 19:12:39.929039 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:39.928997 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 19:12:39.929226 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:39.929044 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm podName:19a0267f-762f-4fb6-971d-f5988a038b5a nodeName:}" failed. No retries permitted until 2026-07-01 19:12:43.929026509 +0000 UTC m=+67.157201044 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rtwv8" (UID: "19a0267f-762f-4fb6-971d-f5988a038b5a") : configmap "kube-rbac-proxy" not found Jul 01 19:12:40.029913 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:40.029880 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:40.030284 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:40.029957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:40.030284 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:40.030033 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 19:12:40.030284 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:40.030069 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle podName:d0c6e2e7-7f8a-4911-a5e1-1933759bb182 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:48.030055422 +0000 UTC m=+71.258229955 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle") pod "router-default-7b759854c7-dwz6b" (UID: "d0c6e2e7-7f8a-4911-a5e1-1933759bb182") : configmap references non-existent config key: service-ca.crt Jul 01 19:12:40.030284 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:40.030094 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs podName:d0c6e2e7-7f8a-4911-a5e1-1933759bb182 nodeName:}" failed. No retries permitted until 2026-07-01 19:12:48.030076667 +0000 UTC m=+71.258251202 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs") pod "router-default-7b759854c7-dwz6b" (UID: "d0c6e2e7-7f8a-4911-a5e1-1933759bb182") : secret "router-metrics-certs-default" not found Jul 01 19:12:42.951669 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:42.951626 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:12:42.953586 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:42.953567 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 19:12:42.962223 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:42.962203 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 01 19:12:42.962301 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:42.962267 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs podName:47beecfe-aae0-485b-b9f4-9c75b22096ea nodeName:}" failed. No retries permitted until 2026-07-01 19:13:46.962247535 +0000 UTC m=+130.190422083 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs") pod "network-metrics-daemon-cbmfg" (UID: "47beecfe-aae0-485b-b9f4-9c75b22096ea") : secret "metrics-daemon-secret" not found Jul 01 19:12:43.052942 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.052900 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbgwq\" (UniqueName: \"kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq\") pod \"network-check-target-pl6zn\" (UID: \"b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83\") " pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:12:43.053065 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.053003 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:12:43.055167 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.055150 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 19:12:43.055231 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.055164 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 19:12:43.065348 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.065321 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 19:12:43.065840 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.065817 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/943fd8e6-36e9-45d0-a60f-78d295d95d9f-original-pull-secret\") pod \"global-pull-secret-syncer-66xvf\" (UID: \"943fd8e6-36e9-45d0-a60f-78d295d95d9f\") " pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:12:43.075957 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.075932 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbgwq\" (UniqueName: \"kubernetes.io/projected/b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83-kube-api-access-sbgwq\") pod \"network-check-target-pl6zn\" (UID: \"b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83\") " pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:12:43.158521 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.158498 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-88jrr\"" Jul 01 19:12:43.166897 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.166880 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:12:43.257477 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.257455 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66xvf" Jul 01 19:12:43.316742 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.316715 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pl6zn"] Jul 01 19:12:43.320104 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:12:43.320079 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3963cc8_9e0d_4e3f_aeb7_7752df7fcc83.slice/crio-81f57c7cea25683cb550d240fb819087cb9dd462b5be076b49bb60f35fc3ec8f WatchSource:0}: Error finding container 81f57c7cea25683cb550d240fb819087cb9dd462b5be076b49bb60f35fc3ec8f: Status 404 returned error can't find the container with id 81f57c7cea25683cb550d240fb819087cb9dd462b5be076b49bb60f35fc3ec8f Jul 01 19:12:43.373863 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.373839 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-66xvf"] Jul 01 19:12:43.376804 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:12:43.376776 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943fd8e6_36e9_45d0_a60f_78d295d95d9f.slice/crio-23f07307c34c92999df3f5fc2a7c331899d6e55190a1bfa847af13919ce5e0d4 WatchSource:0}: Error finding container 23f07307c34c92999df3f5fc2a7c331899d6e55190a1bfa847af13919ce5e0d4: Status 404 returned error can't find the container with id 23f07307c34c92999df3f5fc2a7c331899d6e55190a1bfa847af13919ce5e0d4 Jul 01 19:12:43.516272 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.516189 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-66xvf" event={"ID":"943fd8e6-36e9-45d0-a60f-78d295d95d9f","Type":"ContainerStarted","Data":"23f07307c34c92999df3f5fc2a7c331899d6e55190a1bfa847af13919ce5e0d4"} Jul 01 19:12:43.517161 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.517137 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pl6zn" event={"ID":"b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83","Type":"ContainerStarted","Data":"81f57c7cea25683cb550d240fb819087cb9dd462b5be076b49bb60f35fc3ec8f"} Jul 01 19:12:43.961153 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.961104 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:43.961764 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.961270 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:43.961764 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:43.961269 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 19:12:43.961764 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:43.961320 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:12:43.961764 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:43.961389 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls podName:19a0267f-762f-4fb6-971d-f5988a038b5a nodeName:}" failed. No retries permitted until 2026-07-01 19:12:51.961366071 +0000 UTC m=+75.189540609 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rtwv8" (UID: "19a0267f-762f-4fb6-971d-f5988a038b5a") : secret "insights-runtime-extractor-tls" not found Jul 01 19:12:43.961764 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:43.961310 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 19:12:43.961764 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:43.961432 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 19:12:43.961764 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:43.961479 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm podName:19a0267f-762f-4fb6-971d-f5988a038b5a nodeName:}" failed. No retries permitted until 2026-07-01 19:12:51.961458395 +0000 UTC m=+75.189632941 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rtwv8" (UID: "19a0267f-762f-4fb6-971d-f5988a038b5a") : configmap "kube-rbac-proxy" not found Jul 01 19:12:43.961764 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:43.961502 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls podName:8855e158-eba7-4470-b0be-cce54da3d10b nodeName:}" failed. No retries permitted until 2026-07-01 19:13:15.961491341 +0000 UTC m=+99.189665879 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls") pod "dns-default-rcb2k" (UID: "8855e158-eba7-4470-b0be-cce54da3d10b") : secret "dns-default-metrics-tls" not found Jul 01 19:12:47.999169 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:47.999144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:12:47.999476 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:47.999307 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 19:12:47.999476 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:47.999372 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls podName:6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0 nodeName:}" failed. No retries permitted until 2026-07-01 19:13:03.999350426 +0000 UTC m=+87.227524958 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-vxf6s" (UID: "6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0") : secret "cluster-monitoring-operator-tls" not found Jul 01 19:12:48.099686 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:48.099660 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:48.099791 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:48.099773 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:12:48.099842 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:48.099828 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle podName:d0c6e2e7-7f8a-4911-a5e1-1933759bb182 nodeName:}" failed. No retries permitted until 2026-07-01 19:13:04.099809552 +0000 UTC m=+87.327984085 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle") pod "router-default-7b759854c7-dwz6b" (UID: "d0c6e2e7-7f8a-4911-a5e1-1933759bb182") : configmap references non-existent config key: service-ca.crt Jul 01 19:12:48.099886 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:48.099876 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 19:12:48.099916 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:48.099911 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs podName:d0c6e2e7-7f8a-4911-a5e1-1933759bb182 nodeName:}" failed. No retries permitted until 2026-07-01 19:13:04.099899993 +0000 UTC m=+87.328074531 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs") pod "router-default-7b759854c7-dwz6b" (UID: "d0c6e2e7-7f8a-4911-a5e1-1933759bb182") : secret "router-metrics-certs-default" not found Jul 01 19:12:48.534068 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:48.534033 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-66xvf" event={"ID":"943fd8e6-36e9-45d0-a60f-78d295d95d9f","Type":"ContainerStarted","Data":"c159efa7186013ef77f5ec6904494d1c156dc468d4301a1f9f2f41b2f1bac3ce"} Jul 01 19:12:48.535555 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:48.535520 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pl6zn" event={"ID":"b3963cc8-9e0d-4e3f-aeb7-7752df7fcc83","Type":"ContainerStarted","Data":"ef34d21d431e6ca41dec9111c4c0bea30afbd0c1baf3c00ba881bf0219b36009"} Jul 01 19:12:48.535674 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:48.535649 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:12:48.552074 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:48.552036 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-66xvf" podStartSLOduration=66.939267904 podStartE2EDuration="1m11.55202476s" totalImagesPullingTime="4.612756856s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:12:48.550580699 +0000 UTC m=+71.778755255" watchObservedRunningTime="2026-07-01 19:12:48.55202476 +0000 UTC m=+71.780199314" Jul 01 19:12:48.569070 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:48.569015 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pl6zn" podStartSLOduration=66.911010233 podStartE2EDuration="1m11.568994818s" totalImagesPullingTime="4.657984585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:12:48.568594756 +0000 UTC m=+71.796769311" watchObservedRunningTime="2026-07-01 19:12:48.568994818 +0000 UTC m=+71.797169373" Jul 01 19:12:52.030640 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:52.030590 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:52.031143 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:52.030744 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:12:52.031143 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:52.030861 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 19:12:52.031143 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:12:52.030915 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls podName:19a0267f-762f-4fb6-971d-f5988a038b5a nodeName:}" failed. No retries permitted until 2026-07-01 19:13:08.03090197 +0000 UTC m=+91.259076504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rtwv8" (UID: "19a0267f-762f-4fb6-971d-f5988a038b5a") : secret "insights-runtime-extractor-tls" not found Jul 01 19:12:52.031143 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:12:52.031102 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a0267f-762f-4fb6-971d-f5988a038b5a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:13:04.020823 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.020695 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:13:04.022969 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.022949 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-vxf6s\" (UID: \"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:13:04.122035 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.121999 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:13:04.122230 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.122055 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:13:04.122617 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.122595 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-service-ca-bundle\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:13:04.124235 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.124219 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0c6e2e7-7f8a-4911-a5e1-1933759bb182-metrics-certs\") pod \"router-default-7b759854c7-dwz6b\" (UID: \"d0c6e2e7-7f8a-4911-a5e1-1933759bb182\") " pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:13:04.271799 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.271719 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4sdhh\"" Jul 01 19:13:04.280551 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.280531 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" Jul 01 19:13:04.387785 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.387757 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9v92p\"" Jul 01 19:13:04.396377 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.396359 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:13:04.405745 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.405600 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s"] Jul 01 19:13:04.409759 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:13:04.409729 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dbf42f7_f386_43e5_bfa9_8b9d2a4117f0.slice/crio-5b0b75488b7f0b92a78c30d6509afd1463906ec80566582f02b1eec1e6887385 WatchSource:0}: Error finding container 5b0b75488b7f0b92a78c30d6509afd1463906ec80566582f02b1eec1e6887385: Status 404 returned error can't find the container with id 5b0b75488b7f0b92a78c30d6509afd1463906ec80566582f02b1eec1e6887385 Jul 01 19:13:04.517725 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.517698 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b759854c7-dwz6b"] Jul 01 19:13:04.520341 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:13:04.520318 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c6e2e7_7f8a_4911_a5e1_1933759bb182.slice/crio-a1cf7733e01736211499a8dc9a28c80ab8db263b3121b9cf43a9613c45740623 WatchSource:0}: Error finding container a1cf7733e01736211499a8dc9a28c80ab8db263b3121b9cf43a9613c45740623: Status 404 returned error can't find the container with id a1cf7733e01736211499a8dc9a28c80ab8db263b3121b9cf43a9613c45740623 Jul 01 19:13:04.577422 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.577369 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b759854c7-dwz6b" event={"ID":"d0c6e2e7-7f8a-4911-a5e1-1933759bb182","Type":"ContainerStarted","Data":"a1cf7733e01736211499a8dc9a28c80ab8db263b3121b9cf43a9613c45740623"} Jul 01 19:13:04.578601 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:04.578578 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" event={"ID":"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0","Type":"ContainerStarted","Data":"5b0b75488b7f0b92a78c30d6509afd1463906ec80566582f02b1eec1e6887385"} Jul 01 19:13:05.584503 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:05.584463 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b759854c7-dwz6b" event={"ID":"d0c6e2e7-7f8a-4911-a5e1-1933759bb182","Type":"ContainerStarted","Data":"92918b20cfc15563b3811d8ab0b8c544187d7ffa83129d28f0a8cfca93d232b1"} Jul 01 19:13:05.603388 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:05.603336 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7b759854c7-dwz6b" podStartSLOduration=33.603322281 podStartE2EDuration="33.603322281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:12:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:13:05.60200287 +0000 UTC m=+88.830177427" watchObservedRunningTime="2026-07-01 19:13:05.603322281 +0000 UTC m=+88.831496836" Jul 01 19:13:06.396872 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:06.396837 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:13:06.399269 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:06.399247 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:13:06.589481 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:06.589387 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" event={"ID":"6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0","Type":"ContainerStarted","Data":"f16921f11b9e37307d46e0877364cd5fa30fbb78faf30de8121f5a4375ec9c56"} Jul 01 19:13:06.589892 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:06.589611 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:13:06.590768 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:06.590746 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7b759854c7-dwz6b" Jul 01 19:13:06.606089 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:06.606036 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-vxf6s" podStartSLOduration=32.764241194 podStartE2EDuration="34.606016326s" totalImagesPullingTime="1.841775132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:12:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:13:06.604777767 +0000 UTC m=+89.832952380" watchObservedRunningTime="2026-07-01 19:13:06.606016326 +0000 UTC m=+89.834190882" Jul 01 19:13:08.054001 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:08.053944 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:13:08.056315 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:08.056292 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a0267f-762f-4fb6-971d-f5988a038b5a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rtwv8\" (UID: \"19a0267f-762f-4fb6-971d-f5988a038b5a\") " pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:13:08.298669 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:08.298639 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-54dhx\"" Jul 01 19:13:08.306343 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:08.306282 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rtwv8" Jul 01 19:13:08.425019 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:08.424971 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rtwv8"] Jul 01 19:13:08.428544 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:13:08.428514 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a0267f_762f_4fb6_971d_f5988a038b5a.slice/crio-97d4e1f746c154f18ee4528c303a7089fe1df02aec56f4653bed3b787bb6d94b WatchSource:0}: Error finding container 97d4e1f746c154f18ee4528c303a7089fe1df02aec56f4653bed3b787bb6d94b: Status 404 returned error can't find the container with id 97d4e1f746c154f18ee4528c303a7089fe1df02aec56f4653bed3b787bb6d94b Jul 01 19:13:08.596846 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:08.596760 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rtwv8" event={"ID":"19a0267f-762f-4fb6-971d-f5988a038b5a","Type":"ContainerStarted","Data":"97d4e1f746c154f18ee4528c303a7089fe1df02aec56f4653bed3b787bb6d94b"} Jul 01 19:13:09.443666 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.443631 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6"] Jul 01 19:13:09.445844 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.445821 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:09.448103 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.448078 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 01 19:13:09.449007 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.448988 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 01 19:13:09.449106 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.449027 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 01 19:13:09.453254 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.453234 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 01 19:13:09.466248 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.466225 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6"] Jul 01 19:13:09.547053 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.547021 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-645bcc8fc9-lwtn6"] Jul 01 19:13:09.549220 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.549200 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.554155 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.554131 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 01 19:13:09.554992 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.554806 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 01 19:13:09.554992 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.554853 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 01 19:13:09.558937 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.556563 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qj9pz\"" Jul 01 19:13:09.563167 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.563148 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 01 19:13:09.566865 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.566843 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/035f9be1-3056-4a1f-8c02-e27c298a5ebc-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c7875d4d-22pw6\" (UID: \"035f9be1-3056-4a1f-8c02-e27c298a5ebc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:09.566971 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.566892 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/035f9be1-3056-4a1f-8c02-e27c298a5ebc-tmp\") pod \"klusterlet-addon-workmgr-6c7875d4d-22pw6\" (UID: \"035f9be1-3056-4a1f-8c02-e27c298a5ebc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:09.566971 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.566942 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt8fx\" (UniqueName: \"kubernetes.io/projected/035f9be1-3056-4a1f-8c02-e27c298a5ebc-kube-api-access-kt8fx\") pod \"klusterlet-addon-workmgr-6c7875d4d-22pw6\" (UID: \"035f9be1-3056-4a1f-8c02-e27c298a5ebc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:09.579650 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.579630 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-645bcc8fc9-lwtn6"] Jul 01 19:13:09.612890 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.612861 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-645bcc8fc9-lwtn6"] Jul 01 19:13:09.613096 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:13:09.613072 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-p2sr4 registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-p2sr4 registry-certificates registry-tls trusted-ca]: context canceled" pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" podUID="90765c64-d3fb-495c-b9d8-b24cde840ac2" Jul 01 19:13:09.667439 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.667397 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/035f9be1-3056-4a1f-8c02-e27c298a5ebc-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c7875d4d-22pw6\" (UID: \"035f9be1-3056-4a1f-8c02-e27c298a5ebc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:09.667609 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.667447 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90765c64-d3fb-495c-b9d8-b24cde840ac2-registry-certificates\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.667609 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.667496 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90765c64-d3fb-495c-b9d8-b24cde840ac2-installation-pull-secrets\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.667711 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.667641 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-bound-sa-token\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.667711 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.667677 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/035f9be1-3056-4a1f-8c02-e27c298a5ebc-tmp\") pod \"klusterlet-addon-workmgr-6c7875d4d-22pw6\" (UID: \"035f9be1-3056-4a1f-8c02-e27c298a5ebc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:09.667711 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.667705 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90765c64-d3fb-495c-b9d8-b24cde840ac2-ca-trust-extracted\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.667861 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.667740 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt8fx\" (UniqueName: \"kubernetes.io/projected/035f9be1-3056-4a1f-8c02-e27c298a5ebc-kube-api-access-kt8fx\") pod \"klusterlet-addon-workmgr-6c7875d4d-22pw6\" (UID: \"035f9be1-3056-4a1f-8c02-e27c298a5ebc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:09.667861 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.667792 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90765c64-d3fb-495c-b9d8-b24cde840ac2-image-registry-private-configuration\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.667861 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.667815 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-registry-tls\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.667861 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.667854 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90765c64-d3fb-495c-b9d8-b24cde840ac2-trusted-ca\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.668073 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.667994 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2sr4\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-kube-api-access-p2sr4\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.668073 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.668034 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/035f9be1-3056-4a1f-8c02-e27c298a5ebc-tmp\") pod \"klusterlet-addon-workmgr-6c7875d4d-22pw6\" (UID: \"035f9be1-3056-4a1f-8c02-e27c298a5ebc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:09.670098 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.670075 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/035f9be1-3056-4a1f-8c02-e27c298a5ebc-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c7875d4d-22pw6\" (UID: \"035f9be1-3056-4a1f-8c02-e27c298a5ebc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:09.678693 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.678672 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt8fx\" (UniqueName: \"kubernetes.io/projected/035f9be1-3056-4a1f-8c02-e27c298a5ebc-kube-api-access-kt8fx\") pod \"klusterlet-addon-workmgr-6c7875d4d-22pw6\" (UID: \"035f9be1-3056-4a1f-8c02-e27c298a5ebc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:09.758134 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.758088 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:09.768609 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.768581 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90765c64-d3fb-495c-b9d8-b24cde840ac2-trusted-ca\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.768743 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.768627 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2sr4\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-kube-api-access-p2sr4\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.768743 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.768664 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90765c64-d3fb-495c-b9d8-b24cde840ac2-registry-certificates\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.768743 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.768708 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90765c64-d3fb-495c-b9d8-b24cde840ac2-installation-pull-secrets\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.768743 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.768731 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-bound-sa-token\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.769016 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.768763 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90765c64-d3fb-495c-b9d8-b24cde840ac2-ca-trust-extracted\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.769016 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.768827 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90765c64-d3fb-495c-b9d8-b24cde840ac2-image-registry-private-configuration\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.769016 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.768860 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-registry-tls\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.770221 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.769545 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90765c64-d3fb-495c-b9d8-b24cde840ac2-ca-trust-extracted\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.771616 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.770886 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90765c64-d3fb-495c-b9d8-b24cde840ac2-registry-certificates\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.771616 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.771243 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90765c64-d3fb-495c-b9d8-b24cde840ac2-trusted-ca\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.771785 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.771740 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-registry-tls\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.772167 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.772143 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90765c64-d3fb-495c-b9d8-b24cde840ac2-installation-pull-secrets\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.773563 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.773532 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90765c64-d3fb-495c-b9d8-b24cde840ac2-image-registry-private-configuration\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.779729 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.779704 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2sr4\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-kube-api-access-p2sr4\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.780038 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.780003 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-bound-sa-token\") pod \"image-registry-645bcc8fc9-lwtn6\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:09.892909 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:09.892877 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6"] Jul 01 19:13:10.603476 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.603443 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:10.620925 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.620898 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:10.675241 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.675212 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-bound-sa-token\") pod \"90765c64-d3fb-495c-b9d8-b24cde840ac2\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " Jul 01 19:13:10.675415 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.675249 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-registry-tls\") pod \"90765c64-d3fb-495c-b9d8-b24cde840ac2\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " Jul 01 19:13:10.675415 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.675291 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90765c64-d3fb-495c-b9d8-b24cde840ac2-trusted-ca\") pod \"90765c64-d3fb-495c-b9d8-b24cde840ac2\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " Jul 01 19:13:10.675415 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.675321 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90765c64-d3fb-495c-b9d8-b24cde840ac2-registry-certificates\") pod \"90765c64-d3fb-495c-b9d8-b24cde840ac2\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " Jul 01 19:13:10.675415 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.675368 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2sr4\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-kube-api-access-p2sr4\") pod \"90765c64-d3fb-495c-b9d8-b24cde840ac2\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " Jul 01 19:13:10.675415 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.675413 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90765c64-d3fb-495c-b9d8-b24cde840ac2-image-registry-private-configuration\") pod \"90765c64-d3fb-495c-b9d8-b24cde840ac2\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " Jul 01 19:13:10.675621 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.675442 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90765c64-d3fb-495c-b9d8-b24cde840ac2-ca-trust-extracted\") pod \"90765c64-d3fb-495c-b9d8-b24cde840ac2\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " Jul 01 19:13:10.675621 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.675512 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90765c64-d3fb-495c-b9d8-b24cde840ac2-installation-pull-secrets\") pod \"90765c64-d3fb-495c-b9d8-b24cde840ac2\" (UID: \"90765c64-d3fb-495c-b9d8-b24cde840ac2\") " Jul 01 19:13:10.675728 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.675671 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90765c64-d3fb-495c-b9d8-b24cde840ac2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "90765c64-d3fb-495c-b9d8-b24cde840ac2" (UID: "90765c64-d3fb-495c-b9d8-b24cde840ac2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:13:10.675887 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.675758 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90765c64-d3fb-495c-b9d8-b24cde840ac2-trusted-ca\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:13:10.675991 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.675955 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90765c64-d3fb-495c-b9d8-b24cde840ac2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "90765c64-d3fb-495c-b9d8-b24cde840ac2" (UID: "90765c64-d3fb-495c-b9d8-b24cde840ac2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:13:10.677389 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.677363 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90765c64-d3fb-495c-b9d8-b24cde840ac2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "90765c64-d3fb-495c-b9d8-b24cde840ac2" (UID: "90765c64-d3fb-495c-b9d8-b24cde840ac2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 19:13:10.678014 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.677966 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-kube-api-access-p2sr4" (OuterVolumeSpecName: "kube-api-access-p2sr4") pod "90765c64-d3fb-495c-b9d8-b24cde840ac2" (UID: "90765c64-d3fb-495c-b9d8-b24cde840ac2"). InnerVolumeSpecName "kube-api-access-p2sr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:13:10.678299 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.678246 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90765c64-d3fb-495c-b9d8-b24cde840ac2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "90765c64-d3fb-495c-b9d8-b24cde840ac2" (UID: "90765c64-d3fb-495c-b9d8-b24cde840ac2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:13:10.678589 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.678564 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "90765c64-d3fb-495c-b9d8-b24cde840ac2" (UID: "90765c64-d3fb-495c-b9d8-b24cde840ac2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:13:10.679247 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.679224 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90765c64-d3fb-495c-b9d8-b24cde840ac2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "90765c64-d3fb-495c-b9d8-b24cde840ac2" (UID: "90765c64-d3fb-495c-b9d8-b24cde840ac2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:13:10.679514 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.679492 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "90765c64-d3fb-495c-b9d8-b24cde840ac2" (UID: "90765c64-d3fb-495c-b9d8-b24cde840ac2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:13:10.776557 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.776517 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90765c64-d3fb-495c-b9d8-b24cde840ac2-image-registry-private-configuration\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:13:10.776557 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.776558 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90765c64-d3fb-495c-b9d8-b24cde840ac2-ca-trust-extracted\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:13:10.776761 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.776572 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90765c64-d3fb-495c-b9d8-b24cde840ac2-installation-pull-secrets\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:13:10.776761 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.776583 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-bound-sa-token\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:13:10.776761 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.776593 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-registry-tls\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:13:10.776761 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.776601 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90765c64-d3fb-495c-b9d8-b24cde840ac2-registry-certificates\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:13:10.776761 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:10.776609 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2sr4\" (UniqueName: \"kubernetes.io/projected/90765c64-d3fb-495c-b9d8-b24cde840ac2-kube-api-access-p2sr4\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:13:11.606510 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:11.606434 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-645bcc8fc9-lwtn6" Jul 01 19:13:11.617029 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:13:11.617000 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod035f9be1_3056_4a1f_8c02_e27c298a5ebc.slice/crio-ae915debf409e6527b52a33d16d1279bf794523f392015de8c3e5bee63a4ea8c WatchSource:0}: Error finding container ae915debf409e6527b52a33d16d1279bf794523f392015de8c3e5bee63a4ea8c: Status 404 returned error can't find the container with id ae915debf409e6527b52a33d16d1279bf794523f392015de8c3e5bee63a4ea8c Jul 01 19:13:11.635844 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:11.635813 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-645bcc8fc9-lwtn6"] Jul 01 19:13:11.639543 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:11.639517 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-645bcc8fc9-lwtn6"] Jul 01 19:13:12.612326 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:12.612288 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rtwv8" event={"ID":"19a0267f-762f-4fb6-971d-f5988a038b5a","Type":"ContainerStarted","Data":"9ea98081c142bb3316bc5379b879c82c124bbfaed1b0a43a96ac9f01db9f996f"} Jul 01 19:13:12.613495 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:12.613466 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" event={"ID":"035f9be1-3056-4a1f-8c02-e27c298a5ebc","Type":"ContainerStarted","Data":"ae915debf409e6527b52a33d16d1279bf794523f392015de8c3e5bee63a4ea8c"} Jul 01 19:13:13.251462 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:13.251433 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90765c64-d3fb-495c-b9d8-b24cde840ac2" path="/var/lib/kubelet/pods/90765c64-d3fb-495c-b9d8-b24cde840ac2/volumes" Jul 01 19:13:13.619680 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:13.619596 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rtwv8" event={"ID":"19a0267f-762f-4fb6-971d-f5988a038b5a","Type":"ContainerStarted","Data":"93f8a3b33be5cb35c8388eb733d8dcd88dd15ddd57bac75c51d820d4a0e8d645"} Jul 01 19:13:14.797684 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:14.797647 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-md7rc"] Jul 01 19:13:14.800916 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:14.800861 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:14.804129 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:14.804092 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 01 19:13:14.804233 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:14.804140 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-g78sq\"" Jul 01 19:13:14.804707 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:14.804615 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 01 19:13:14.804998 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:14.804845 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 01 19:13:14.818913 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:14.818884 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-md7rc"] Jul 01 19:13:14.906400 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:14.906368 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8g5j\" (UniqueName: \"kubernetes.io/projected/ef6ec9c8-c755-4724-97ba-128f7f3f8e8d-kube-api-access-g8g5j\") pod \"prometheus-operator-7b7d896497-md7rc\" (UID: \"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:14.906561 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:14.906408 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef6ec9c8-c755-4724-97ba-128f7f3f8e8d-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-md7rc\" (UID: \"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:14.906561 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:14.906435 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef6ec9c8-c755-4724-97ba-128f7f3f8e8d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-md7rc\" (UID: \"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:14.906561 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:14.906509 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef6ec9c8-c755-4724-97ba-128f7f3f8e8d-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-md7rc\" (UID: \"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:15.007741 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:15.007706 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef6ec9c8-c755-4724-97ba-128f7f3f8e8d-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-md7rc\" (UID: \"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:15.007929 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:15.007802 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8g5j\" (UniqueName: \"kubernetes.io/projected/ef6ec9c8-c755-4724-97ba-128f7f3f8e8d-kube-api-access-g8g5j\") pod \"prometheus-operator-7b7d896497-md7rc\" (UID: \"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:15.007929 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:15.007833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef6ec9c8-c755-4724-97ba-128f7f3f8e8d-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-md7rc\" (UID: \"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:15.007929 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:15.007865 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef6ec9c8-c755-4724-97ba-128f7f3f8e8d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-md7rc\" (UID: \"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:15.008583 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:15.008556 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef6ec9c8-c755-4724-97ba-128f7f3f8e8d-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-md7rc\" (UID: \"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:15.010690 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:15.010662 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef6ec9c8-c755-4724-97ba-128f7f3f8e8d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-md7rc\" (UID: \"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:15.010783 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:15.010708 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef6ec9c8-c755-4724-97ba-128f7f3f8e8d-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-md7rc\" (UID: \"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:15.016221 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:15.016192 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8g5j\" (UniqueName: \"kubernetes.io/projected/ef6ec9c8-c755-4724-97ba-128f7f3f8e8d-kube-api-access-g8g5j\") pod \"prometheus-operator-7b7d896497-md7rc\" (UID: \"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:15.112366 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:15.112299 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" Jul 01 19:13:16.015184 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:16.015155 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:13:16.017771 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:16.017744 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8855e158-eba7-4470-b0be-cce54da3d10b-metrics-tls\") pod \"dns-default-rcb2k\" (UID: \"8855e158-eba7-4470-b0be-cce54da3d10b\") " pod="openshift-dns/dns-default-rcb2k" Jul 01 19:13:16.304922 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:16.304850 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6ddhd\"" Jul 01 19:13:16.313642 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:16.313615 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rcb2k" Jul 01 19:13:17.103309 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:17.103282 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-md7rc"] Jul 01 19:13:17.106838 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:13:17.106785 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef6ec9c8_c755_4724_97ba_128f7f3f8e8d.slice/crio-cca6d2449375abfc08e7ae76125e09f0f52ba9059c6c014109c024511903ccf3 WatchSource:0}: Error finding container cca6d2449375abfc08e7ae76125e09f0f52ba9059c6c014109c024511903ccf3: Status 404 returned error can't find the container with id cca6d2449375abfc08e7ae76125e09f0f52ba9059c6c014109c024511903ccf3 Jul 01 19:13:17.120802 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:17.120781 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rcb2k"] Jul 01 19:13:17.124073 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:13:17.124029 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8855e158_eba7_4470_b0be_cce54da3d10b.slice/crio-1ed8bec35cdfed32c37b8a6d485758c26076591e6f725688b64a21108dc54ecb WatchSource:0}: Error finding container 1ed8bec35cdfed32c37b8a6d485758c26076591e6f725688b64a21108dc54ecb: Status 404 returned error can't find the container with id 1ed8bec35cdfed32c37b8a6d485758c26076591e6f725688b64a21108dc54ecb Jul 01 19:13:17.634363 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:17.634276 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" event={"ID":"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d","Type":"ContainerStarted","Data":"cca6d2449375abfc08e7ae76125e09f0f52ba9059c6c014109c024511903ccf3"} Jul 01 19:13:17.636212 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:17.635981 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" event={"ID":"035f9be1-3056-4a1f-8c02-e27c298a5ebc","Type":"ContainerStarted","Data":"0af8f2c44193fa6e7f6bb4a0b338f59215608f7482d6ea8d400fce679fd4bcb5"} Jul 01 19:13:17.636212 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:17.636150 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:17.638366 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:17.638347 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" Jul 01 19:13:17.638681 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:17.638655 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rtwv8" event={"ID":"19a0267f-762f-4fb6-971d-f5988a038b5a","Type":"ContainerStarted","Data":"02069c1eb3fe9e6f08750aa980a04357c18caeb73a79c98bd652f1d8e6b2f7fa"} Jul 01 19:13:17.639957 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:17.639933 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rcb2k" event={"ID":"8855e158-eba7-4470-b0be-cce54da3d10b","Type":"ContainerStarted","Data":"1ed8bec35cdfed32c37b8a6d485758c26076591e6f725688b64a21108dc54ecb"} Jul 01 19:13:17.661276 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:17.661229 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c7875d4d-22pw6" podStartSLOduration=3.296731158 podStartE2EDuration="8.661211124s" totalImagesPullingTime="5.364479966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:13:17.660142606 +0000 UTC m=+100.888317163" watchObservedRunningTime="2026-07-01 19:13:17.661211124 +0000 UTC m=+100.889385680" Jul 01 19:13:17.680779 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:17.680725 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rtwv8" podStartSLOduration=33.259305903 podStartE2EDuration="41.680697825s" totalImagesPullingTime="8.421391922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:12:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:13:17.680004563 +0000 UTC m=+100.908179122" watchObservedRunningTime="2026-07-01 19:13:17.680697825 +0000 UTC m=+100.908872383" Jul 01 19:13:19.541170 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:19.541138 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pl6zn" Jul 01 19:13:19.648624 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:19.648592 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rcb2k" event={"ID":"8855e158-eba7-4470-b0be-cce54da3d10b","Type":"ContainerStarted","Data":"b9263c4a4465643bfc8582c3723e2d6fa647102c39be11981279e347af1e5093"} Jul 01 19:13:19.648786 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:19.648655 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rcb2k" event={"ID":"8855e158-eba7-4470-b0be-cce54da3d10b","Type":"ContainerStarted","Data":"cb8363d561dceadf79c7d6fee86a83ed3271e55d4d8eef561bae288aff1b92c8"} Jul 01 19:13:19.648786 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:19.648686 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rcb2k" Jul 01 19:13:19.650247 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:19.650222 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" event={"ID":"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d","Type":"ContainerStarted","Data":"a7d027386aa351a750292ff05d3e2e8d42a6a8929afd7eb10b8b344a33e56e82"} Jul 01 19:13:19.650374 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:19.650287 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" event={"ID":"ef6ec9c8-c755-4724-97ba-128f7f3f8e8d","Type":"ContainerStarted","Data":"d80c6e2ef552e57e20be36aec112df38ae1c0a384759640dee7ed397174f9f75"} Jul 01 19:13:19.665429 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:19.665388 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rcb2k" podStartSLOduration=65.831461882 podStartE2EDuration="1m7.665374847s" totalImagesPullingTime="1.833912965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:13:19.664260277 +0000 UTC m=+102.892434832" watchObservedRunningTime="2026-07-01 19:13:19.665374847 +0000 UTC m=+102.893549401" Jul 01 19:13:19.679320 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:19.679277 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-md7rc" podStartSLOduration=3.828458891 podStartE2EDuration="5.679261742s" totalImagesPullingTime="1.850802851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:13:19.678358994 +0000 UTC m=+102.906533551" watchObservedRunningTime="2026-07-01 19:13:19.679261742 +0000 UTC m=+102.907436298" Jul 01 19:13:21.145606 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.145569 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-r9rll"] Jul 01 19:13:21.175035 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.175003 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-r9rll"] Jul 01 19:13:21.175192 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.175138 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.177710 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.177683 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 01 19:13:21.177710 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.177700 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 01 19:13:21.177890 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.177712 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 01 19:13:21.177890 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.177704 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-tnvfp\"" Jul 01 19:13:21.179583 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.179564 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qjn5l"] Jul 01 19:13:21.200938 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.200919 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.202928 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.202912 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 01 19:13:21.203027 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.202938 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 01 19:13:21.203027 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.203013 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cx6h2\"" Jul 01 19:13:21.203108 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.203074 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 01 19:13:21.257458 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.257432 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/801580e2-ea20-4b49-bd1b-95248917be1c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.257578 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.257466 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/801580e2-ea20-4b49-bd1b-95248917be1c-metrics-client-ca\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.257578 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.257521 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/801580e2-ea20-4b49-bd1b-95248917be1c-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.257578 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.257574 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7vpq\" (UniqueName: \"kubernetes.io/projected/801580e2-ea20-4b49-bd1b-95248917be1c-kube-api-access-l7vpq\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.257683 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.257594 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/801580e2-ea20-4b49-bd1b-95248917be1c-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.257683 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.257611 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/801580e2-ea20-4b49-bd1b-95248917be1c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.357904 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.357873 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/801580e2-ea20-4b49-bd1b-95248917be1c-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.358046 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.357907 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-tls\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.358046 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.357960 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7vpq\" (UniqueName: \"kubernetes.io/projected/801580e2-ea20-4b49-bd1b-95248917be1c-kube-api-access-l7vpq\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.358046 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.357989 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-accelerators-collector-config\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.358046 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.358020 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/801580e2-ea20-4b49-bd1b-95248917be1c-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.358220 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.358047 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-sys\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.358220 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.358198 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-wtmp\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.358294 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.358241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/801580e2-ea20-4b49-bd1b-95248917be1c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.358294 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.358273 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.358381 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.358305 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-metrics-client-ca\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.358381 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.358360 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/801580e2-ea20-4b49-bd1b-95248917be1c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.358479 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.358421 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-root\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.358479 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.358450 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/801580e2-ea20-4b49-bd1b-95248917be1c-metrics-client-ca\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.358588 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.358479 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dknm\" (UniqueName: \"kubernetes.io/projected/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-kube-api-access-8dknm\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.358588 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.358501 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/801580e2-ea20-4b49-bd1b-95248917be1c-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.358588 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.358522 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-textfile\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.360043 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.359683 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/801580e2-ea20-4b49-bd1b-95248917be1c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.360043 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.359771 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/801580e2-ea20-4b49-bd1b-95248917be1c-metrics-client-ca\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.361014 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.360988 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/801580e2-ea20-4b49-bd1b-95248917be1c-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.361247 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.361218 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/801580e2-ea20-4b49-bd1b-95248917be1c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.366652 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.366620 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7vpq\" (UniqueName: \"kubernetes.io/projected/801580e2-ea20-4b49-bd1b-95248917be1c-kube-api-access-l7vpq\") pod \"kube-state-metrics-845b954455-r9rll\" (UID: \"801580e2-ea20-4b49-bd1b-95248917be1c\") " pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.459580 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.459558 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dknm\" (UniqueName: \"kubernetes.io/projected/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-kube-api-access-8dknm\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.459756 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.459589 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-textfile\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.459756 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.459616 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-tls\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.459890 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:13:21.459760 2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 01 19:13:21.459890 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:13:21.459822 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-tls podName:193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb nodeName:}" failed. No retries permitted until 2026-07-01 19:13:21.959806964 +0000 UTC m=+105.187981497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-tls") pod "node-exporter-qjn5l" (UID: "193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb") : secret "node-exporter-tls" not found Jul 01 19:13:21.459890 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.459836 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-accelerators-collector-config\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.459890 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.459856 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-sys\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.459890 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.459872 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-wtmp\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.460151 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.459901 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.460151 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.459923 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-metrics-client-ca\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.460151 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.459932 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-textfile\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.460151 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.459953 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-root\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.460151 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.459918 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-sys\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.460151 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.460001 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-root\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.460151 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.460031 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-wtmp\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.460357 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.460283 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-accelerators-collector-config\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.460469 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.460448 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-metrics-client-ca\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.462080 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.462065 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.468413 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.468395 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dknm\" (UniqueName: \"kubernetes.io/projected/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-kube-api-access-8dknm\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.484284 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.484262 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" Jul 01 19:13:21.637714 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.637685 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-r9rll"] Jul 01 19:13:21.640931 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:13:21.640902 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod801580e2_ea20_4b49_bd1b_95248917be1c.slice/crio-7d78868ece4e9338e9215e9af784d65d56d9c1109e3336ee24605c794d740b5a WatchSource:0}: Error finding container 7d78868ece4e9338e9215e9af784d65d56d9c1109e3336ee24605c794d740b5a: Status 404 returned error can't find the container with id 7d78868ece4e9338e9215e9af784d65d56d9c1109e3336ee24605c794d740b5a Jul 01 19:13:21.658341 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.658309 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" event={"ID":"801580e2-ea20-4b49-bd1b-95248917be1c","Type":"ContainerStarted","Data":"7d78868ece4e9338e9215e9af784d65d56d9c1109e3336ee24605c794d740b5a"} Jul 01 19:13:21.963415 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.963387 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-tls\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:21.965568 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:21.965542 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb-node-exporter-tls\") pod \"node-exporter-qjn5l\" (UID: \"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb\") " pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:22.109657 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:22.109628 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qjn5l" Jul 01 19:13:22.150847 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:13:22.150813 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod193cc7a8_c0bb_49b5_8bfb_19b19e8c48eb.slice/crio-c470070540d531bc5a729d7226b2eb90c6980db2381307e11b583caa2f14c02c WatchSource:0}: Error finding container c470070540d531bc5a729d7226b2eb90c6980db2381307e11b583caa2f14c02c: Status 404 returned error can't find the container with id c470070540d531bc5a729d7226b2eb90c6980db2381307e11b583caa2f14c02c Jul 01 19:13:22.665452 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:22.665410 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qjn5l" event={"ID":"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb","Type":"ContainerStarted","Data":"c470070540d531bc5a729d7226b2eb90c6980db2381307e11b583caa2f14c02c"} Jul 01 19:13:24.674287 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:24.674243 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" event={"ID":"801580e2-ea20-4b49-bd1b-95248917be1c","Type":"ContainerStarted","Data":"e19d5a13e247dfef6597b33c5506a046b6a25f945ee2e3821c9c2d88d500f6bb"} Jul 01 19:13:24.674669 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:24.674313 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" event={"ID":"801580e2-ea20-4b49-bd1b-95248917be1c","Type":"ContainerStarted","Data":"8376cc0edfe7f1f4b0626d69528b2c1f2145383f0d50277cfc71244ccbdade31"} Jul 01 19:13:24.674669 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:24.674341 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" event={"ID":"801580e2-ea20-4b49-bd1b-95248917be1c","Type":"ContainerStarted","Data":"40b56bf19e876c93d6e2a4cf649ae277c16729a9d03e2c410b37b375d9b55e88"} Jul 01 19:13:24.675840 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:24.675816 2577 generic.go:356] "Generic (PLEG): container finished" podID="193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb" containerID="b3cf7df1eb66218b881e97a1d5c38ad2d4e7501d689f94134460ccc33d67aeca" exitCode=0 Jul 01 19:13:24.675942 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:24.675871 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qjn5l" event={"ID":"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb","Type":"ContainerDied","Data":"b3cf7df1eb66218b881e97a1d5c38ad2d4e7501d689f94134460ccc33d67aeca"} Jul 01 19:13:24.691744 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:24.691702 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-r9rll" podStartSLOduration=1.426980163 podStartE2EDuration="3.691686924s" totalImagesPullingTime="2.264706761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:13:24.690452943 +0000 UTC m=+107.918627500" watchObservedRunningTime="2026-07-01 19:13:24.691686924 +0000 UTC m=+107.919861479" Jul 01 19:13:25.446133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.446093 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-644dbb586b-s6r75"] Jul 01 19:13:25.448476 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.448458 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.450559 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.450537 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 01 19:13:25.451199 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.451180 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 19:13:25.451299 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.451216 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-brpxm\"" Jul 01 19:13:25.451448 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.451434 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 01 19:13:25.451533 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.451449 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-15k5v9ueq6ta\"" Jul 01 19:13:25.451533 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.451501 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 01 19:13:25.459705 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.459685 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-644dbb586b-s6r75"] Jul 01 19:13:25.590020 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.589988 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.590227 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.590045 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-secret-metrics-server-client-certs\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.590227 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.590073 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-client-ca-bundle\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.590227 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.590088 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-audit-log\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.590227 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.590133 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-metrics-server-audit-profiles\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.590417 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.590237 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqkrv\" (UniqueName: \"kubernetes.io/projected/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-kube-api-access-jqkrv\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.590417 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.590293 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-secret-metrics-server-tls\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.685038 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.685001 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qjn5l" event={"ID":"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb","Type":"ContainerStarted","Data":"883b0481d4d5db25201d8493d6e51e34f7d59c75aa370e50562d03645e5e7aed"} Jul 01 19:13:25.685424 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.685072 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qjn5l" event={"ID":"193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb","Type":"ContainerStarted","Data":"97301f6299f403d862c3dd3da7b9615326b3fc47407350966b0a27e5130a6304"} Jul 01 19:13:25.691505 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.691484 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-secret-metrics-server-client-certs\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.691581 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.691513 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-client-ca-bundle\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.691581 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.691530 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-audit-log\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.691691 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.691673 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-metrics-server-audit-profiles\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.691750 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.691725 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqkrv\" (UniqueName: \"kubernetes.io/projected/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-kube-api-access-jqkrv\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.691847 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.691831 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-audit-log\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.691926 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.691826 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-secret-metrics-server-tls\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.691926 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.691904 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.692570 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.692538 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.692676 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.692587 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-metrics-server-audit-profiles\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.694073 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.694053 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-client-ca-bundle\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.694200 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.694137 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-secret-metrics-server-client-certs\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.694261 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.694215 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-secret-metrics-server-tls\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.700538 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.700487 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqkrv\" (UniqueName: \"kubernetes.io/projected/cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7-kube-api-access-jqkrv\") pod \"metrics-server-644dbb586b-s6r75\" (UID: \"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7\") " pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.703798 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.703755 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qjn5l" podStartSLOduration=2.871457649 podStartE2EDuration="4.703741303s" totalImagesPullingTime="1.752570313s" totalInitContainerRuntime="79.713341ms" isStatefulPod=true podCreationTimestamp="2026-07-01 19:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:13:25.703699373 +0000 UTC m=+108.931873929" watchObservedRunningTime="2026-07-01 19:13:25.703741303 +0000 UTC m=+108.931915870" Jul 01 19:13:25.757921 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.757888 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:25.892047 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.892014 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-644dbb586b-s6r75"] Jul 01 19:13:25.896515 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:13:25.896490 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc9f4385_41d2_40ff_96ce_d1bbd2d1b0a7.slice/crio-369a69b531971ac08ccbdcbcb60772ad418c2986c2816a2fce97203a5bcf0dc6 WatchSource:0}: Error finding container 369a69b531971ac08ccbdcbcb60772ad418c2986c2816a2fce97203a5bcf0dc6: Status 404 returned error can't find the container with id 369a69b531971ac08ccbdcbcb60772ad418c2986c2816a2fce97203a5bcf0dc6 Jul 01 19:13:25.915630 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.915601 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q"] Jul 01 19:13:25.918526 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.918507 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" Jul 01 19:13:25.920376 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.920354 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 01 19:13:25.920632 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.920615 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-7gmjc\"" Jul 01 19:13:25.926778 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.926755 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q"] Jul 01 19:13:25.994708 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.994682 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b4b2bad-d026-423e-bbed-fa190bdafa3b-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-jjn9q\" (UID: \"2b4b2bad-d026-423e-bbed-fa190bdafa3b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" Jul 01 19:13:25.994892 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:25.994727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjsh4\" (UniqueName: \"kubernetes.io/projected/2b4b2bad-d026-423e-bbed-fa190bdafa3b-kube-api-access-cjsh4\") pod \"monitoring-plugin-cd8656b8-jjn9q\" (UID: \"2b4b2bad-d026-423e-bbed-fa190bdafa3b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" Jul 01 19:13:26.095852 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:26.095814 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b4b2bad-d026-423e-bbed-fa190bdafa3b-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-jjn9q\" (UID: \"2b4b2bad-d026-423e-bbed-fa190bdafa3b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" Jul 01 19:13:26.096005 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:26.095860 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjsh4\" (UniqueName: \"kubernetes.io/projected/2b4b2bad-d026-423e-bbed-fa190bdafa3b-kube-api-access-cjsh4\") pod \"monitoring-plugin-cd8656b8-jjn9q\" (UID: \"2b4b2bad-d026-423e-bbed-fa190bdafa3b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" Jul 01 19:13:26.096005 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:13:26.095967 2577 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 01 19:13:26.096145 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:13:26.096029 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b4b2bad-d026-423e-bbed-fa190bdafa3b-monitoring-plugin-cert podName:2b4b2bad-d026-423e-bbed-fa190bdafa3b nodeName:}" failed. No retries permitted until 2026-07-01 19:13:26.596014155 +0000 UTC m=+109.824188688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/2b4b2bad-d026-423e-bbed-fa190bdafa3b-monitoring-plugin-cert") pod "monitoring-plugin-cd8656b8-jjn9q" (UID: "2b4b2bad-d026-423e-bbed-fa190bdafa3b") : secret "monitoring-plugin-cert" not found Jul 01 19:13:26.106582 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:26.106554 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjsh4\" (UniqueName: \"kubernetes.io/projected/2b4b2bad-d026-423e-bbed-fa190bdafa3b-kube-api-access-cjsh4\") pod \"monitoring-plugin-cd8656b8-jjn9q\" (UID: \"2b4b2bad-d026-423e-bbed-fa190bdafa3b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" Jul 01 19:13:26.598997 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:26.598964 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b4b2bad-d026-423e-bbed-fa190bdafa3b-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-jjn9q\" (UID: \"2b4b2bad-d026-423e-bbed-fa190bdafa3b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" Jul 01 19:13:26.601309 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:26.601290 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b4b2bad-d026-423e-bbed-fa190bdafa3b-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-jjn9q\" (UID: \"2b4b2bad-d026-423e-bbed-fa190bdafa3b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" Jul 01 19:13:26.689564 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:26.689526 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" event={"ID":"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7","Type":"ContainerStarted","Data":"369a69b531971ac08ccbdcbcb60772ad418c2986c2816a2fce97203a5bcf0dc6"} Jul 01 19:13:26.828639 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:26.828605 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" Jul 01 19:13:26.959252 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:26.959219 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q"] Jul 01 19:13:26.963125 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:13:26.963085 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b4b2bad_d026_423e_bbed_fa190bdafa3b.slice/crio-b783eb98378699713d0eed8ed3f77103d0397e4f4f496d75117d3b4f444fd1d5 WatchSource:0}: Error finding container b783eb98378699713d0eed8ed3f77103d0397e4f4f496d75117d3b4f444fd1d5: Status 404 returned error can't find the container with id b783eb98378699713d0eed8ed3f77103d0397e4f4f496d75117d3b4f444fd1d5 Jul 01 19:13:27.388807 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.388765 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:13:27.392122 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.392086 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.395901 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.395875 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 19:13:27.396031 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.396006 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 19:13:27.396099 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.396011 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 19:13:27.396204 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.396172 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 19:13:27.397961 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.397934 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 19:13:27.398095 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.397934 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d25c06c6j2t2v\"" Jul 01 19:13:27.398401 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.398378 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 19:13:27.398973 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.398808 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 19:13:27.398973 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.398887 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 19:13:27.398973 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.398914 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 19:13:27.399968 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.399937 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 19:13:27.400648 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.400229 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 19:13:27.400648 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.400362 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 19:13:27.400648 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.400515 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-w527x\"" Jul 01 19:13:27.403057 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.403030 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 19:13:27.407823 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.407803 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 19:13:27.409440 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.409078 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:13:27.510558 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510521 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.510756 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510569 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.510756 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510625 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.510756 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510656 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88e44ec9-6951-4fdc-a725-0c8567d248b8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.510756 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510681 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-web-config\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.510756 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510706 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.510756 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510740 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511039 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510808 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511039 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510833 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-config\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511039 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510858 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88e44ec9-6951-4fdc-a725-0c8567d248b8-config-out\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511039 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510884 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511039 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510913 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511039 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.510946 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511039 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.511008 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t92k5\" (UniqueName: \"kubernetes.io/projected/88e44ec9-6951-4fdc-a725-0c8567d248b8-kube-api-access-t92k5\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511312 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.511072 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511312 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.511101 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511312 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.511140 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511312 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.511164 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511312 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.511190 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.511312 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.511204 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612462 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88e44ec9-6951-4fdc-a725-0c8567d248b8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612507 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-web-config\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612534 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612573 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612606 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612630 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-config\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612651 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88e44ec9-6951-4fdc-a725-0c8567d248b8-config-out\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612674 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612699 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612729 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612767 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t92k5\" (UniqueName: \"kubernetes.io/projected/88e44ec9-6951-4fdc-a725-0c8567d248b8-kube-api-access-t92k5\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612802 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612831 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.613133 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612858 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.614001 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612880 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.614001 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612921 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.614001 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612943 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.614001 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.612971 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.614001 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.613001 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.614001 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.613049 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.614001 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.613304 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.615225 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.614613 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.615493 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.615467 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88e44ec9-6951-4fdc-a725-0c8567d248b8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.615859 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.615736 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.617004 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.616167 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.617004 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.616293 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.617004 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.616951 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.617622 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.617390 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.617732 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.617626 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.618508 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.618484 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.620568 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.618861 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-web-config\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.620568 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.618915 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-config\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.620568 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.618999 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.620568 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.619800 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.620568 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.620224 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88e44ec9-6951-4fdc-a725-0c8567d248b8-config-out\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.621726 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.621680 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.621822 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.621759 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.622077 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.622042 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.622247 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.622227 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.623428 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.623407 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t92k5\" (UniqueName: \"kubernetes.io/projected/88e44ec9-6951-4fdc-a725-0c8567d248b8-kube-api-access-t92k5\") pod \"prometheus-k8s-0\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.695668 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.695561 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" event={"ID":"cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7","Type":"ContainerStarted","Data":"7c9ceb0802b15bb0c423dc9fa8915efd2bc822a89f008b323f92e44acf0a22df"} Jul 01 19:13:27.696783 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.696749 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" event={"ID":"2b4b2bad-d026-423e-bbed-fa190bdafa3b","Type":"ContainerStarted","Data":"b783eb98378699713d0eed8ed3f77103d0397e4f4f496d75117d3b4f444fd1d5"} Jul 01 19:13:27.707798 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.707769 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:27.734170 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.734026 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" podStartSLOduration=1.082336111 podStartE2EDuration="2.734001529s" totalImagesPullingTime="1.651665418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:13:27.732899699 +0000 UTC m=+110.961074270" watchObservedRunningTime="2026-07-01 19:13:27.734001529 +0000 UTC m=+110.962176087" Jul 01 19:13:27.880520 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:27.880483 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:13:27.884534 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:13:27.884479 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88e44ec9_6951_4fdc_a725_0c8567d248b8.slice/crio-1194aa0b42a311fa887a18efda69002c78080d96293e800e0b43b87ec739c41e WatchSource:0}: Error finding container 1194aa0b42a311fa887a18efda69002c78080d96293e800e0b43b87ec739c41e: Status 404 returned error can't find the container with id 1194aa0b42a311fa887a18efda69002c78080d96293e800e0b43b87ec739c41e Jul 01 19:13:28.701745 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:28.701716 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerStarted","Data":"1194aa0b42a311fa887a18efda69002c78080d96293e800e0b43b87ec739c41e"} Jul 01 19:13:28.703088 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:28.703064 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" event={"ID":"2b4b2bad-d026-423e-bbed-fa190bdafa3b","Type":"ContainerStarted","Data":"0e41568a1ee2fc63fd626d874f928b44a0390a9b57ddb2122419c9c25f5d9fe3"} Jul 01 19:13:28.722560 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:28.722521 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" podStartSLOduration=2.067225427 podStartE2EDuration="3.722506738s" totalImagesPullingTime="1.655281311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:13:28.721229369 +0000 UTC m=+111.949403931" watchObservedRunningTime="2026-07-01 19:13:28.722506738 +0000 UTC m=+111.950681293" Jul 01 19:13:29.656424 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:29.656351 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rcb2k" Jul 01 19:13:29.708537 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:29.708508 2577 generic.go:356] "Generic (PLEG): container finished" podID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerID="07273c71ab9e6b07d413190abad9061d25d47e928e9072268c026bb6d55f6973" exitCode=0 Jul 01 19:13:29.708972 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:29.708596 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerDied","Data":"07273c71ab9e6b07d413190abad9061d25d47e928e9072268c026bb6d55f6973"} Jul 01 19:13:29.708972 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:29.708816 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" Jul 01 19:13:29.714224 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:29.714202 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-jjn9q" Jul 01 19:13:33.727614 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:33.727564 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerStarted","Data":"992bf5d5280854e3c1211b5a906563fc22d327861eb738f5f2e28cd015c64bfb"} Jul 01 19:13:33.727965 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:33.727700 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerStarted","Data":"aa030defbff20eb6a2f77545069503f7967fe9ec3ac9bc885294f15f60b54c76"} Jul 01 19:13:36.741298 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:36.741266 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerStarted","Data":"ad39d6625f861801d98799015d2312722dd6e9e730536e683150b97818b0d694"} Jul 01 19:13:36.741665 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:36.741328 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerStarted","Data":"2705b110b4a8bf910c1c29d6c8450b11f98df356cd4d62d7554fc9f18219f7b6"} Jul 01 19:13:36.741665 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:36.741354 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerStarted","Data":"3faa03f46ddc459fa53b0ffa8aa47bbcff5845b12c17a8dd13665908ac2a4463"} Jul 01 19:13:36.741665 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:36.741378 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerStarted","Data":"5b1442ee1db0772ec51ca3dca3a59333cd2bdb590e0527fae3b50b4f92822436"} Jul 01 19:13:36.774254 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:36.774191 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.955159322 podStartE2EDuration="9.774169721s" totalImagesPullingTime="7.79814759s" totalInitContainerRuntime="20.862809ms" isStatefulPod=false podCreationTimestamp="2026-07-01 19:13:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:13:36.771765135 +0000 UTC m=+119.999939690" watchObservedRunningTime="2026-07-01 19:13:36.774169721 +0000 UTC m=+120.002344277" Jul 01 19:13:37.708893 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:37.708862 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:13:45.758902 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:45.758866 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:45.759429 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:45.758936 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:13:46.978763 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:46.978726 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:13:46.980998 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:46.980976 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47beecfe-aae0-485b-b9f4-9c75b22096ea-metrics-certs\") pod \"network-metrics-daemon-cbmfg\" (UID: \"47beecfe-aae0-485b-b9f4-9c75b22096ea\") " pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:13:47.167438 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:47.167403 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9wqsv\"" Jul 01 19:13:47.175392 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:47.175372 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbmfg" Jul 01 19:13:47.391203 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:47.391175 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cbmfg"] Jul 01 19:13:47.393990 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:13:47.393964 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47beecfe_aae0_485b_b9f4_9c75b22096ea.slice/crio-90bf9d0b008604e9fa49b766173e18b2a21d72046bc12a9ea8e8d2dbe072ab27 WatchSource:0}: Error finding container 90bf9d0b008604e9fa49b766173e18b2a21d72046bc12a9ea8e8d2dbe072ab27: Status 404 returned error can't find the container with id 90bf9d0b008604e9fa49b766173e18b2a21d72046bc12a9ea8e8d2dbe072ab27 Jul 01 19:13:47.784459 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:47.784415 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cbmfg" event={"ID":"47beecfe-aae0-485b-b9f4-9c75b22096ea","Type":"ContainerStarted","Data":"90bf9d0b008604e9fa49b766173e18b2a21d72046bc12a9ea8e8d2dbe072ab27"} Jul 01 19:13:49.793414 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:49.793374 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cbmfg" event={"ID":"47beecfe-aae0-485b-b9f4-9c75b22096ea","Type":"ContainerStarted","Data":"aba0bb97da943584dc3af3ea5381a00647907dd8b3b9aba553c78c17c16239c0"} Jul 01 19:13:49.793773 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:49.793447 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cbmfg" event={"ID":"47beecfe-aae0-485b-b9f4-9c75b22096ea","Type":"ContainerStarted","Data":"c45b6c792779381a6f8cf61eed28eca08bea4a7362ee4186bb61293eef7dd944"} Jul 01 19:13:49.809009 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:13:49.808961 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cbmfg" podStartSLOduration=131.441630288 podStartE2EDuration="2m12.808945106s" totalImagesPullingTime="1.367314818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:13:49.807521156 +0000 UTC m=+133.035695710" watchObservedRunningTime="2026-07-01 19:13:49.808945106 +0000 UTC m=+133.037119660" Jul 01 19:14:05.763959 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:05.763931 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:14:05.767798 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:05.767780 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-644dbb586b-s6r75" Jul 01 19:14:27.708943 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:27.708905 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:27.724331 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:27.724305 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:27.947204 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:27.947172 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:45.765106 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:45.764994 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:14:45.765550 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:45.765387 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="prometheus" containerID="cri-o://aa030defbff20eb6a2f77545069503f7967fe9ec3ac9bc885294f15f60b54c76" gracePeriod=600 Jul 01 19:14:45.765550 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:45.765443 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="kube-rbac-proxy-web" containerID="cri-o://3faa03f46ddc459fa53b0ffa8aa47bbcff5845b12c17a8dd13665908ac2a4463" gracePeriod=600 Jul 01 19:14:45.765550 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:45.765439 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ad39d6625f861801d98799015d2312722dd6e9e730536e683150b97818b0d694" gracePeriod=600 Jul 01 19:14:45.765550 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:45.765482 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="thanos-sidecar" containerID="cri-o://5b1442ee1db0772ec51ca3dca3a59333cd2bdb590e0527fae3b50b4f92822436" gracePeriod=600 Jul 01 19:14:45.765550 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:45.765526 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="kube-rbac-proxy" containerID="cri-o://2705b110b4a8bf910c1c29d6c8450b11f98df356cd4d62d7554fc9f18219f7b6" gracePeriod=600 Jul 01 19:14:45.765775 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:45.765553 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="config-reloader" containerID="cri-o://992bf5d5280854e3c1211b5a906563fc22d327861eb738f5f2e28cd015c64bfb" gracePeriod=600 Jul 01 19:14:46.004427 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.004392 2577 generic.go:356] "Generic (PLEG): container finished" podID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerID="ad39d6625f861801d98799015d2312722dd6e9e730536e683150b97818b0d694" exitCode=0 Jul 01 19:14:46.004427 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.004419 2577 generic.go:356] "Generic (PLEG): container finished" podID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerID="2705b110b4a8bf910c1c29d6c8450b11f98df356cd4d62d7554fc9f18219f7b6" exitCode=0 Jul 01 19:14:46.004427 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.004427 2577 generic.go:356] "Generic (PLEG): container finished" podID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerID="3faa03f46ddc459fa53b0ffa8aa47bbcff5845b12c17a8dd13665908ac2a4463" exitCode=0 Jul 01 19:14:46.004427 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.004433 2577 generic.go:356] "Generic (PLEG): container finished" podID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerID="5b1442ee1db0772ec51ca3dca3a59333cd2bdb590e0527fae3b50b4f92822436" exitCode=0 Jul 01 19:14:46.004427 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.004440 2577 generic.go:356] "Generic (PLEG): container finished" podID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerID="992bf5d5280854e3c1211b5a906563fc22d327861eb738f5f2e28cd015c64bfb" exitCode=0 Jul 01 19:14:46.004427 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.004446 2577 generic.go:356] "Generic (PLEG): container finished" podID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerID="aa030defbff20eb6a2f77545069503f7967fe9ec3ac9bc885294f15f60b54c76" exitCode=0 Jul 01 19:14:46.004889 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.004470 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerDied","Data":"ad39d6625f861801d98799015d2312722dd6e9e730536e683150b97818b0d694"} Jul 01 19:14:46.004889 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.004568 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerDied","Data":"2705b110b4a8bf910c1c29d6c8450b11f98df356cd4d62d7554fc9f18219f7b6"} Jul 01 19:14:46.004889 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.004604 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerDied","Data":"3faa03f46ddc459fa53b0ffa8aa47bbcff5845b12c17a8dd13665908ac2a4463"} Jul 01 19:14:46.004889 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.004630 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerDied","Data":"5b1442ee1db0772ec51ca3dca3a59333cd2bdb590e0527fae3b50b4f92822436"} Jul 01 19:14:46.004889 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.004656 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerDied","Data":"992bf5d5280854e3c1211b5a906563fc22d327861eb738f5f2e28cd015c64bfb"} Jul 01 19:14:46.004889 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.004686 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerDied","Data":"aa030defbff20eb6a2f77545069503f7967fe9ec3ac9bc885294f15f60b54c76"} Jul 01 19:14:46.042409 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.042380 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:46.059613 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.059586 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-trusted-ca-bundle\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.059613 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.059621 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t92k5\" (UniqueName: \"kubernetes.io/projected/88e44ec9-6951-4fdc-a725-0c8567d248b8-kube-api-access-t92k5\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.059843 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.059641 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-0\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.059843 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.059667 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88e44ec9-6951-4fdc-a725-0c8567d248b8-tls-assets\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.059843 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.059694 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.059843 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.059724 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-config\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060057 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.059753 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060131 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060100 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-web-config\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060189 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060161 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-metrics-client-ca\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060189 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060185 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-db\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060290 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060220 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-1\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060290 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060247 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-2\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060290 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060272 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-serving-certs-ca-bundle\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060434 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060301 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-kube-rbac-proxy\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060434 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060331 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-thanos-prometheus-http-client-file\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060434 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060356 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88e44ec9-6951-4fdc-a725-0c8567d248b8-config-out\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060434 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060387 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-metrics-client-certs\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060434 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060432 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-grpc-tls\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060686 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060473 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-tls\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.060686 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060519 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-kubelet-serving-ca-bundle\") pod \"88e44ec9-6951-4fdc-a725-0c8567d248b8\" (UID: \"88e44ec9-6951-4fdc-a725-0c8567d248b8\") " Jul 01 19:14:46.064136 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060027 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:14:46.064136 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.060800 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:14:46.064136 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.061145 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:14:46.064136 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.061789 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:14:46.064136 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.063459 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:14:46.064136 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.063518 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:14:46.064136 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.063879 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:14:46.064564 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.064519 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:14:46.064873 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.064843 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:14:46.065564 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.065531 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:14:46.066032 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.066004 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 19:14:46.066225 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.066182 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:14:46.066486 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.066412 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-config" (OuterVolumeSpecName: "config") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:14:46.066486 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.066410 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88e44ec9-6951-4fdc-a725-0c8567d248b8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:14:46.066486 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.066450 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88e44ec9-6951-4fdc-a725-0c8567d248b8-kube-api-access-t92k5" (OuterVolumeSpecName: "kube-api-access-t92k5") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "kube-api-access-t92k5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:14:46.066692 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.066483 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:14:46.066885 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.066855 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:14:46.067171 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.067147 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:14:46.067815 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.067791 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88e44ec9-6951-4fdc-a725-0c8567d248b8-config-out" (OuterVolumeSpecName: "config-out") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 19:14:46.076146 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.076106 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-web-config" (OuterVolumeSpecName: "web-config") pod "88e44ec9-6951-4fdc-a725-0c8567d248b8" (UID: "88e44ec9-6951-4fdc-a725-0c8567d248b8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:14:46.161520 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161491 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t92k5\" (UniqueName: \"kubernetes.io/projected/88e44ec9-6951-4fdc-a725-0c8567d248b8-kube-api-access-t92k5\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161520 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161517 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161532 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88e44ec9-6951-4fdc-a725-0c8567d248b8-tls-assets\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161546 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161555 2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-config\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161563 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161571 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-web-config\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161579 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-metrics-client-ca\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161587 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-db\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161600 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161608 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161617 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161626 2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-kube-rbac-proxy\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161635 2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161645 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88e44ec9-6951-4fdc-a725-0c8567d248b8-config-out\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161653 2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-metrics-client-certs\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161663 2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-grpc-tls\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161676 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88e44ec9-6951-4fdc-a725-0c8567d248b8-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161686 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:46.161737 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:46.161694 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e44ec9-6951-4fdc-a725-0c8567d248b8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:14:47.010793 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.010756 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88e44ec9-6951-4fdc-a725-0c8567d248b8","Type":"ContainerDied","Data":"1194aa0b42a311fa887a18efda69002c78080d96293e800e0b43b87ec739c41e"} Jul 01 19:14:47.011196 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.010808 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.011196 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.010865 2577 scope.go:117] "RemoveContainer" containerID="ad39d6625f861801d98799015d2312722dd6e9e730536e683150b97818b0d694" Jul 01 19:14:47.018838 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.018814 2577 scope.go:117] "RemoveContainer" containerID="2705b110b4a8bf910c1c29d6c8450b11f98df356cd4d62d7554fc9f18219f7b6" Jul 01 19:14:47.025770 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.025750 2577 scope.go:117] "RemoveContainer" containerID="3faa03f46ddc459fa53b0ffa8aa47bbcff5845b12c17a8dd13665908ac2a4463" Jul 01 19:14:47.032444 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.032424 2577 scope.go:117] "RemoveContainer" containerID="5b1442ee1db0772ec51ca3dca3a59333cd2bdb590e0527fae3b50b4f92822436" Jul 01 19:14:47.038375 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.038351 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:14:47.039578 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.039561 2577 scope.go:117] "RemoveContainer" containerID="992bf5d5280854e3c1211b5a906563fc22d327861eb738f5f2e28cd015c64bfb" Jul 01 19:14:47.042770 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.042750 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:14:47.046422 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.046398 2577 scope.go:117] "RemoveContainer" containerID="aa030defbff20eb6a2f77545069503f7967fe9ec3ac9bc885294f15f60b54c76" Jul 01 19:14:47.053211 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.053194 2577 scope.go:117] "RemoveContainer" containerID="07273c71ab9e6b07d413190abad9061d25d47e928e9072268c026bb6d55f6973" Jul 01 19:14:47.069243 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069224 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:14:47.069496 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069484 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="config-reloader" Jul 01 19:14:47.069540 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069498 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="config-reloader" Jul 01 19:14:47.069540 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069508 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="kube-rbac-proxy" Jul 01 19:14:47.069540 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069513 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="kube-rbac-proxy" Jul 01 19:14:47.069540 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069525 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="kube-rbac-proxy-web" Jul 01 19:14:47.069540 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069531 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="kube-rbac-proxy-web" Jul 01 19:14:47.069540 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069538 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="thanos-sidecar" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069543 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="thanos-sidecar" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069551 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="kube-rbac-proxy-thanos" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069557 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="kube-rbac-proxy-thanos" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069563 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="init-config-reloader" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069568 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="init-config-reloader" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069573 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="prometheus" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069577 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="prometheus" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069631 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="thanos-sidecar" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069638 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="kube-rbac-proxy" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069644 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="config-reloader" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069649 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="kube-rbac-proxy-web" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069654 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="kube-rbac-proxy-thanos" Jul 01 19:14:47.069713 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.069660 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" containerName="prometheus" Jul 01 19:14:47.073773 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.073756 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.075799 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.075778 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 19:14:47.076526 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.076508 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 19:14:47.076606 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.076573 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 19:14:47.076820 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.076801 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 19:14:47.076933 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.076874 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 19:14:47.077045 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.077033 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 19:14:47.077182 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.077165 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d25c06c6j2t2v\"" Jul 01 19:14:47.077268 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.077165 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 19:14:47.077268 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.077250 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 19:14:47.077580 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.077525 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 19:14:47.077580 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.077543 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-w527x\"" Jul 01 19:14:47.077580 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.077548 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 19:14:47.077580 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.077557 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 19:14:47.077921 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.077596 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 19:14:47.079501 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.079477 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 19:14:47.083344 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.083327 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 19:14:47.088547 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.088529 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:14:47.167681 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167649 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.167681 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167684 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.167871 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167705 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.167871 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167732 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.167871 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167752 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.167871 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167768 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f1662f3-0094-44fc-aead-a1159a6856c7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.167871 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167786 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.167871 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167852 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.168053 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167880 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.168053 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167911 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.168053 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167927 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-config\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.168053 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167942 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.168053 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.167975 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.168053 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.168002 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.168053 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.168021 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.168477 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.168073 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc8kk\" (UniqueName: \"kubernetes.io/projected/2f1662f3-0094-44fc-aead-a1159a6856c7-kube-api-access-pc8kk\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.168477 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.168100 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f1662f3-0094-44fc-aead-a1159a6856c7-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.168477 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.168139 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.168477 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.168153 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.168477 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.168167 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.254947 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.254914 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88e44ec9-6951-4fdc-a725-0c8567d248b8" path="/var/lib/kubelet/pods/88e44ec9-6951-4fdc-a725-0c8567d248b8/volumes" Jul 01 19:14:47.268434 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268408 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.268586 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268446 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-config\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.268586 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268541 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.268705 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.268705 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268639 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.268705 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268681 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.268866 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268739 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc8kk\" (UniqueName: \"kubernetes.io/projected/2f1662f3-0094-44fc-aead-a1159a6856c7-kube-api-access-pc8kk\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.268866 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268765 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f1662f3-0094-44fc-aead-a1159a6856c7-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.268866 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268788 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.268866 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.268866 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268830 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.269096 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268883 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.269096 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268913 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.269096 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268931 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.269096 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.268945 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.269342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.269319 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.269457 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.269405 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.270239 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.270216 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.270659 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.270632 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.271437 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.271414 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-config\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.271714 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.271691 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.271786 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.271725 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.271786 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.271777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.271895 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.271806 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.271895 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.271836 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f1662f3-0094-44fc-aead-a1159a6856c7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.271895 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.271866 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.272045 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.271918 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.272045 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.272017 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.272045 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.272026 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.272224 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.272145 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.273685 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.272731 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.273685 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.272905 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.273685 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.273315 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.274749 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.274092 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f1662f3-0094-44fc-aead-a1159a6856c7-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.275224 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.275145 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.275224 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.275200 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.275474 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.275455 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f1662f3-0094-44fc-aead-a1159a6856c7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.275564 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.275547 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f1662f3-0094-44fc-aead-a1159a6856c7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.275807 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.275788 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f1662f3-0094-44fc-aead-a1159a6856c7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.279536 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.279516 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc8kk\" (UniqueName: \"kubernetes.io/projected/2f1662f3-0094-44fc-aead-a1159a6856c7-kube-api-access-pc8kk\") pod \"prometheus-k8s-0\" (UID: \"2f1662f3-0094-44fc-aead-a1159a6856c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.384754 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.384668 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:14:47.520077 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:47.520044 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:14:47.523169 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:14:47.523142 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f1662f3_0094_44fc_aead_a1159a6856c7.slice/crio-0e186457eede57bb33a179747a795f43af375e5448c415d67b618d921f442b3f WatchSource:0}: Error finding container 0e186457eede57bb33a179747a795f43af375e5448c415d67b618d921f442b3f: Status 404 returned error can't find the container with id 0e186457eede57bb33a179747a795f43af375e5448c415d67b618d921f442b3f Jul 01 19:14:48.017525 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:48.017492 2577 generic.go:356] "Generic (PLEG): container finished" podID="2f1662f3-0094-44fc-aead-a1159a6856c7" containerID="38afec23d3f01577d73d5fa790e5291fc139612eff6f753ab2e1d5cd5c6bfe61" exitCode=0 Jul 01 19:14:48.017890 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:48.017555 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f1662f3-0094-44fc-aead-a1159a6856c7","Type":"ContainerDied","Data":"38afec23d3f01577d73d5fa790e5291fc139612eff6f753ab2e1d5cd5c6bfe61"} Jul 01 19:14:48.017890 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:48.017627 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f1662f3-0094-44fc-aead-a1159a6856c7","Type":"ContainerStarted","Data":"0e186457eede57bb33a179747a795f43af375e5448c415d67b618d921f442b3f"} Jul 01 19:14:49.024594 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:49.024554 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f1662f3-0094-44fc-aead-a1159a6856c7","Type":"ContainerStarted","Data":"c9f0bd5485e231291374dcd3662bc628b2ef2b3f578efe3dce32d8a17cafbf9a"} Jul 01 19:14:49.024974 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:49.024643 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f1662f3-0094-44fc-aead-a1159a6856c7","Type":"ContainerStarted","Data":"528529b039a9c64401d27ee868af38e67edb260ed485878c511f24ab3169d826"} Jul 01 19:14:49.024974 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:49.024674 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f1662f3-0094-44fc-aead-a1159a6856c7","Type":"ContainerStarted","Data":"a63d92c4f6b0005e538adf34b574ce8f531f9fbf7927a0fbfd8b320e27c71829"} Jul 01 19:14:49.024974 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:49.024698 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f1662f3-0094-44fc-aead-a1159a6856c7","Type":"ContainerStarted","Data":"3ec7c9aee23fa9dc2b8b663494f14c4423270f361ef20ce8d6efb5205169706c"} Jul 01 19:14:49.024974 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:49.024720 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f1662f3-0094-44fc-aead-a1159a6856c7","Type":"ContainerStarted","Data":"778bb711e8187814cc8d279b4a2da9fb7148af8e97f9163607cdbb31dff0acba"} Jul 01 19:14:49.024974 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:49.024743 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f1662f3-0094-44fc-aead-a1159a6856c7","Type":"ContainerStarted","Data":"e152272d93722721ca686f1297d0c8b4573b0732b1db9669fd10f55c86e9e58a"} Jul 01 19:14:49.053782 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:49.053725 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.034201965 podStartE2EDuration="2.053704538s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.502573ms" isStatefulPod=false podCreationTimestamp="2026-07-01 19:14:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:14:49.052304258 +0000 UTC m=+192.280478814" watchObservedRunningTime="2026-07-01 19:14:49.053704538 +0000 UTC m=+192.281879099" Jul 01 19:14:52.385013 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:14:52.384979 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:15:47.385267 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:15:47.385235 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:15:47.400192 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:15:47.400162 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:15:48.254467 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:15:48.254417 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:16:37.169983 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:16:37.169950 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmnn_f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6/ovn-acl-logging/0.log" Jul 01 19:16:37.170586 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:16:37.170087 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmnn_f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6/ovn-acl-logging/0.log" Jul 01 19:16:37.176755 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:16:37.176737 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 01 19:17:44.178048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.177968 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb"] Jul 01 19:17:44.180480 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.180460 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.183393 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.183373 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-grp2q\"" Jul 01 19:17:44.183948 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.183932 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 01 19:17:44.184044 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.184019 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 01 19:17:44.184134 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.184058 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 01 19:17:44.190477 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.190461 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 01 19:17:44.216091 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.216066 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb"] Jul 01 19:17:44.282762 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.282713 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2437657-336f-463e-8d52-8adda08ff3cf-tmp\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.282762 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.282765 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2437657-336f-463e-8d52-8adda08ff3cf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.283018 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.282792 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2437657-336f-463e-8d52-8adda08ff3cf-webhook-cert\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.283018 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.282856 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2437657-336f-463e-8d52-8adda08ff3cf-manifests\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.283018 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.282949 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxztf\" (UniqueName: \"kubernetes.io/projected/c2437657-336f-463e-8d52-8adda08ff3cf-kube-api-access-gxztf\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.384086 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.384051 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2437657-336f-463e-8d52-8adda08ff3cf-webhook-cert\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.384238 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.384099 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2437657-336f-463e-8d52-8adda08ff3cf-manifests\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.384238 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.384181 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxztf\" (UniqueName: \"kubernetes.io/projected/c2437657-336f-463e-8d52-8adda08ff3cf-kube-api-access-gxztf\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.384238 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.384214 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2437657-336f-463e-8d52-8adda08ff3cf-tmp\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.384238 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.384233 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2437657-336f-463e-8d52-8adda08ff3cf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.384573 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.384552 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2437657-336f-463e-8d52-8adda08ff3cf-manifests\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.386511 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.386488 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2437657-336f-463e-8d52-8adda08ff3cf-tmp\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.386721 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.386702 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2437657-336f-463e-8d52-8adda08ff3cf-webhook-cert\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.386809 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.386789 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2437657-336f-463e-8d52-8adda08ff3cf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.398092 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.398073 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxztf\" (UniqueName: \"kubernetes.io/projected/c2437657-336f-463e-8d52-8adda08ff3cf-kube-api-access-gxztf\") pod \"opendatahub-operator-controller-manager-85b6b744df-fjhzb\" (UID: \"c2437657-336f-463e-8d52-8adda08ff3cf\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.490774 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.490739 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:44.645342 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.645259 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb"] Jul 01 19:17:44.649803 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:17:44.649781 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2437657_336f_463e_8d52_8adda08ff3cf.slice/crio-dd61e0b4b4bd666ee2f1b87dacdfaa442efb4eeddbb7c6aaf1e3399799f6b7e0 WatchSource:0}: Error finding container dd61e0b4b4bd666ee2f1b87dacdfaa442efb4eeddbb7c6aaf1e3399799f6b7e0: Status 404 returned error can't find the container with id dd61e0b4b4bd666ee2f1b87dacdfaa442efb4eeddbb7c6aaf1e3399799f6b7e0 Jul 01 19:17:44.651451 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:44.651433 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 19:17:45.657348 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:45.657299 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" event={"ID":"c2437657-336f-463e-8d52-8adda08ff3cf","Type":"ContainerStarted","Data":"dd61e0b4b4bd666ee2f1b87dacdfaa442efb4eeddbb7c6aaf1e3399799f6b7e0"} Jul 01 19:17:47.668176 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:47.668136 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" event={"ID":"c2437657-336f-463e-8d52-8adda08ff3cf","Type":"ContainerStarted","Data":"05d118120609309739cacdb6e4b2670027e36e8395f5e86898e8fd94c021a365"} Jul 01 19:17:48.673743 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:48.673711 2577 generic.go:356] "Generic (PLEG): container finished" podID="c2437657-336f-463e-8d52-8adda08ff3cf" containerID="05d118120609309739cacdb6e4b2670027e36e8395f5e86898e8fd94c021a365" exitCode=0 Jul 01 19:17:48.674145 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:48.673808 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" event={"ID":"c2437657-336f-463e-8d52-8adda08ff3cf","Type":"ContainerDied","Data":"05d118120609309739cacdb6e4b2670027e36e8395f5e86898e8fd94c021a365"} Jul 01 19:17:49.681417 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:49.681378 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" event={"ID":"c2437657-336f-463e-8d52-8adda08ff3cf","Type":"ContainerStarted","Data":"1a30e1dc94bb39b1ae9d51561fa793491b8123b9aab6abafcaab7c9e9a2ac0ef"} Jul 01 19:17:49.681993 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:49.681565 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:17:49.710805 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:17:49.710749 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" podStartSLOduration=1.532001951 podStartE2EDuration="5.710732549s" totalImagesPullingTime="3.273396925s" totalInitContainerRuntime="905.333673ms" isStatefulPod=false podCreationTimestamp="2026-07-01 19:17:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:17:49.70875852 +0000 UTC m=+372.936933074" watchObservedRunningTime="2026-07-01 19:17:49.710732549 +0000 UTC m=+372.938907176" Jul 01 19:18:00.687866 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:00.687832 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85b6b744df-fjhzb" Jul 01 19:18:04.319685 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.319649 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx"] Jul 01 19:18:04.326184 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.326154 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.329423 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.329395 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 01 19:18:04.329578 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.329426 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 01 19:18:04.329578 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.329495 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-dmxjt\"" Jul 01 19:18:04.329578 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.329427 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 01 19:18:04.329729 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.329714 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 01 19:18:04.334549 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.334529 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 01 19:18:04.356681 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.356648 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv5fh\" (UniqueName: \"kubernetes.io/projected/dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996-kube-api-access-xv5fh\") pod \"lws-controller-manager-6858c4896d-hlzqx\" (UID: \"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996\") " pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.356869 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.356717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996-metrics-cert\") pod \"lws-controller-manager-6858c4896d-hlzqx\" (UID: \"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996\") " pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.356869 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.356795 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996-manager-config\") pod \"lws-controller-manager-6858c4896d-hlzqx\" (UID: \"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996\") " pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.356869 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.356866 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996-cert\") pod \"lws-controller-manager-6858c4896d-hlzqx\" (UID: \"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996\") " pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.359698 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.359676 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx"] Jul 01 19:18:04.457827 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.457790 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996-cert\") pod \"lws-controller-manager-6858c4896d-hlzqx\" (UID: \"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996\") " pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.458021 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.457847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv5fh\" (UniqueName: \"kubernetes.io/projected/dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996-kube-api-access-xv5fh\") pod \"lws-controller-manager-6858c4896d-hlzqx\" (UID: \"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996\") " pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.458021 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.457879 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996-metrics-cert\") pod \"lws-controller-manager-6858c4896d-hlzqx\" (UID: \"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996\") " pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.458021 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.457911 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996-manager-config\") pod \"lws-controller-manager-6858c4896d-hlzqx\" (UID: \"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996\") " pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.458740 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.458715 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996-manager-config\") pod \"lws-controller-manager-6858c4896d-hlzqx\" (UID: \"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996\") " pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.460361 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.460342 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996-metrics-cert\") pod \"lws-controller-manager-6858c4896d-hlzqx\" (UID: \"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996\") " pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.460449 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.460387 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996-cert\") pod \"lws-controller-manager-6858c4896d-hlzqx\" (UID: \"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996\") " pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.548103 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.548068 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv5fh\" (UniqueName: \"kubernetes.io/projected/dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996-kube-api-access-xv5fh\") pod \"lws-controller-manager-6858c4896d-hlzqx\" (UID: \"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996\") " pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.636389 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.636290 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:04.862820 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:04.862782 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx"] Jul 01 19:18:04.865926 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:18:04.865894 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbfc6a7c_66b5_4db8_b7a6_7ba8da9c9996.slice/crio-44ce44a150017f90418350abf12adba3073141df19f4d70a04645de90c3a147f WatchSource:0}: Error finding container 44ce44a150017f90418350abf12adba3073141df19f4d70a04645de90c3a147f: Status 404 returned error can't find the container with id 44ce44a150017f90418350abf12adba3073141df19f4d70a04645de90c3a147f Jul 01 19:18:05.085048 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.085014 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-658989848c-6bsd6"] Jul 01 19:18:05.088070 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.088056 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" Jul 01 19:18:05.092235 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.092215 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 01 19:18:05.100483 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.100464 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 01 19:18:05.164363 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.164330 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dcfecef-d3d5-48ff-af17-4a4cabb63c28-tls-certs\") pod \"kube-auth-proxy-658989848c-6bsd6\" (UID: \"4dcfecef-d3d5-48ff-af17-4a4cabb63c28\") " pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" Jul 01 19:18:05.164539 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.164393 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dcfecef-d3d5-48ff-af17-4a4cabb63c28-tmp\") pod \"kube-auth-proxy-658989848c-6bsd6\" (UID: \"4dcfecef-d3d5-48ff-af17-4a4cabb63c28\") " pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" Jul 01 19:18:05.164539 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.164462 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4j5c\" (UniqueName: \"kubernetes.io/projected/4dcfecef-d3d5-48ff-af17-4a4cabb63c28-kube-api-access-v4j5c\") pod \"kube-auth-proxy-658989848c-6bsd6\" (UID: \"4dcfecef-d3d5-48ff-af17-4a4cabb63c28\") " pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" Jul 01 19:18:05.225937 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.225895 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-658989848c-6bsd6"] Jul 01 19:18:05.265891 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.265855 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dcfecef-d3d5-48ff-af17-4a4cabb63c28-tls-certs\") pod \"kube-auth-proxy-658989848c-6bsd6\" (UID: \"4dcfecef-d3d5-48ff-af17-4a4cabb63c28\") " pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" Jul 01 19:18:05.266073 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.265925 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dcfecef-d3d5-48ff-af17-4a4cabb63c28-tmp\") pod \"kube-auth-proxy-658989848c-6bsd6\" (UID: \"4dcfecef-d3d5-48ff-af17-4a4cabb63c28\") " pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" Jul 01 19:18:05.266073 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.266003 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4j5c\" (UniqueName: \"kubernetes.io/projected/4dcfecef-d3d5-48ff-af17-4a4cabb63c28-kube-api-access-v4j5c\") pod \"kube-auth-proxy-658989848c-6bsd6\" (UID: \"4dcfecef-d3d5-48ff-af17-4a4cabb63c28\") " pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" Jul 01 19:18:05.268352 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.268325 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dcfecef-d3d5-48ff-af17-4a4cabb63c28-tmp\") pod \"kube-auth-proxy-658989848c-6bsd6\" (UID: \"4dcfecef-d3d5-48ff-af17-4a4cabb63c28\") " pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" Jul 01 19:18:05.268994 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.268974 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dcfecef-d3d5-48ff-af17-4a4cabb63c28-tls-certs\") pod \"kube-auth-proxy-658989848c-6bsd6\" (UID: \"4dcfecef-d3d5-48ff-af17-4a4cabb63c28\") " pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" Jul 01 19:18:05.280479 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.280458 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4j5c\" (UniqueName: \"kubernetes.io/projected/4dcfecef-d3d5-48ff-af17-4a4cabb63c28-kube-api-access-v4j5c\") pod \"kube-auth-proxy-658989848c-6bsd6\" (UID: \"4dcfecef-d3d5-48ff-af17-4a4cabb63c28\") " pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" Jul 01 19:18:05.397689 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.397601 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" Jul 01 19:18:05.562858 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.562802 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-658989848c-6bsd6"] Jul 01 19:18:05.567443 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:18:05.567394 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dcfecef_d3d5_48ff_af17_4a4cabb63c28.slice/crio-f81b3855d1b780de98c4cbf5bac7822163e680d7faaf08761c2084140bcd3fa6 WatchSource:0}: Error finding container f81b3855d1b780de98c4cbf5bac7822163e680d7faaf08761c2084140bcd3fa6: Status 404 returned error can't find the container with id f81b3855d1b780de98c4cbf5bac7822163e680d7faaf08761c2084140bcd3fa6 Jul 01 19:18:05.749399 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.749352 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" event={"ID":"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996","Type":"ContainerStarted","Data":"44ce44a150017f90418350abf12adba3073141df19f4d70a04645de90c3a147f"} Jul 01 19:18:05.750844 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:05.750805 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" event={"ID":"4dcfecef-d3d5-48ff-af17-4a4cabb63c28","Type":"ContainerStarted","Data":"f81b3855d1b780de98c4cbf5bac7822163e680d7faaf08761c2084140bcd3fa6"} Jul 01 19:18:08.768533 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:08.768487 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" event={"ID":"dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996","Type":"ContainerStarted","Data":"6f7df0b0ad61290e85d5d67ca8a06f569eed55b4367cd6b38ddea167b71d493c"} Jul 01 19:18:08.769016 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:08.768604 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:08.809605 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:08.809549 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" podStartSLOduration=1.7915981570000001 podStartE2EDuration="4.809533127s" totalImagesPullingTime="3.01793497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:18:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:18:08.80784435 +0000 UTC m=+392.036018959" watchObservedRunningTime="2026-07-01 19:18:08.809533127 +0000 UTC m=+392.037707684" Jul 01 19:18:09.774177 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:09.774134 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" event={"ID":"4dcfecef-d3d5-48ff-af17-4a4cabb63c28","Type":"ContainerStarted","Data":"d260cea72f2cd49c54775810bdc64fd36e552553d11637e73de4901d2e462cae"} Jul 01 19:18:09.820754 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:09.820690 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-658989848c-6bsd6" podStartSLOduration=1.498967138 podStartE2EDuration="4.820668801s" totalImagesPullingTime="3.321701663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:18:09.81464465 +0000 UTC m=+393.042819206" watchObservedRunningTime="2026-07-01 19:18:09.820668801 +0000 UTC m=+393.048843358" Jul 01 19:18:19.396584 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.396551 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42"] Jul 01 19:18:19.400137 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.400094 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.404042 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.404019 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 01 19:18:19.406924 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.406907 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7x9vk\"" Jul 01 19:18:19.471913 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.471882 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42"] Jul 01 19:18:19.497191 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.497156 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.497370 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.497201 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.497370 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.497264 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.497370 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.497287 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbbmx\" (UniqueName: \"kubernetes.io/projected/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-kube-api-access-kbbmx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.497370 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.497306 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.497370 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.497336 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.497370 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.497364 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.497566 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.497386 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.497566 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.497418 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.598158 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.598085 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.598158 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.598156 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.598418 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.598181 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.598418 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.598203 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.598418 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.598227 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.598418 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.598301 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.598418 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.598358 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.598418 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.598404 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.598707 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.598433 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbbmx\" (UniqueName: \"kubernetes.io/projected/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-kube-api-access-kbbmx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.598764 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.598735 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.598858 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.598833 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.599039 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.599019 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.599157 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.599108 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.599231 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.599216 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.600684 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.600666 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.601083 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.601063 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.650333 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.650255 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.652322 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.652297 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbbmx\" (UniqueName: \"kubernetes.io/projected/829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f-kube-api-access-kbbmx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49hl42\" (UID: \"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.709460 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.709431 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:19.776401 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.776366 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6858c4896d-hlzqx" Jul 01 19:18:19.879806 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:19.879759 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42"] Jul 01 19:18:19.882757 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:18:19.882730 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod829721a6_f175_4f7b_b0d7_4d3f4b2e1a9f.slice/crio-1982f3f748a80273b193aa0f0def9295b53df65e099d63aa922b76d8e843f8a0 WatchSource:0}: Error finding container 1982f3f748a80273b193aa0f0def9295b53df65e099d63aa922b76d8e843f8a0: Status 404 returned error can't find the container with id 1982f3f748a80273b193aa0f0def9295b53df65e099d63aa922b76d8e843f8a0 Jul 01 19:18:20.824269 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:20.824234 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" event={"ID":"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f","Type":"ContainerStarted","Data":"1982f3f748a80273b193aa0f0def9295b53df65e099d63aa922b76d8e843f8a0"} Jul 01 19:18:23.337855 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:23.337819 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 19:18:23.338168 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:23.337891 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 19:18:23.338168 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:23.337920 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 19:18:23.841510 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:23.841473 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" event={"ID":"829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f","Type":"ContainerStarted","Data":"392eafa1ec0c4b96954c62efab74b2c6d5e4faad71b9800f866c744be4f907e3"} Jul 01 19:18:23.861182 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:23.860989 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" podStartSLOduration=1.40892715 podStartE2EDuration="4.860965414s" totalImagesPullingTime="3.452038264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:18:23.860149875 +0000 UTC m=+407.088324421" watchObservedRunningTime="2026-07-01 19:18:23.860965414 +0000 UTC m=+407.089139969" Jul 01 19:18:24.710172 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:24.710134 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:24.714741 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:24.714714 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:24.845569 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:24.845534 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:18:24.846432 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:18:24.846414 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49hl42" Jul 01 19:19:56.183806 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.183774 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5"] Jul 01 19:19:56.187181 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.187160 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" Jul 01 19:19:56.189486 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.189449 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 01 19:19:56.189608 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.189498 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 01 19:19:56.189608 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.189499 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 01 19:19:56.189716 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.189698 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 01 19:19:56.191058 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.191040 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pnstg\"" Jul 01 19:19:56.201457 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.201437 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5"] Jul 01 19:19:56.212273 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.212244 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vh7h\" (UniqueName: \"kubernetes.io/projected/06a79981-dcc1-457b-bf83-200a86c08d63-kube-api-access-8vh7h\") pod \"kuadrant-console-plugin-66b5774985-dprc5\" (UID: \"06a79981-dcc1-457b-bf83-200a86c08d63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" Jul 01 19:19:56.212397 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.212292 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06a79981-dcc1-457b-bf83-200a86c08d63-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dprc5\" (UID: \"06a79981-dcc1-457b-bf83-200a86c08d63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" Jul 01 19:19:56.212397 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.212347 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/06a79981-dcc1-457b-bf83-200a86c08d63-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dprc5\" (UID: \"06a79981-dcc1-457b-bf83-200a86c08d63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" Jul 01 19:19:56.313561 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.313517 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vh7h\" (UniqueName: \"kubernetes.io/projected/06a79981-dcc1-457b-bf83-200a86c08d63-kube-api-access-8vh7h\") pod \"kuadrant-console-plugin-66b5774985-dprc5\" (UID: \"06a79981-dcc1-457b-bf83-200a86c08d63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" Jul 01 19:19:56.313786 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.313578 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06a79981-dcc1-457b-bf83-200a86c08d63-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dprc5\" (UID: \"06a79981-dcc1-457b-bf83-200a86c08d63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" Jul 01 19:19:56.313786 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.313608 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/06a79981-dcc1-457b-bf83-200a86c08d63-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dprc5\" (UID: \"06a79981-dcc1-457b-bf83-200a86c08d63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" Jul 01 19:19:56.313786 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:19:56.313743 2577 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 01 19:19:56.314021 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:19:56.313800 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a79981-dcc1-457b-bf83-200a86c08d63-plugin-serving-cert podName:06a79981-dcc1-457b-bf83-200a86c08d63 nodeName:}" failed. No retries permitted until 2026-07-01 19:19:56.813784739 +0000 UTC m=+500.041959272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/06a79981-dcc1-457b-bf83-200a86c08d63-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-dprc5" (UID: "06a79981-dcc1-457b-bf83-200a86c08d63") : secret "plugin-serving-cert" not found Jul 01 19:19:56.314304 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.314281 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06a79981-dcc1-457b-bf83-200a86c08d63-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dprc5\" (UID: \"06a79981-dcc1-457b-bf83-200a86c08d63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" Jul 01 19:19:56.323516 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.323488 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vh7h\" (UniqueName: \"kubernetes.io/projected/06a79981-dcc1-457b-bf83-200a86c08d63-kube-api-access-8vh7h\") pod \"kuadrant-console-plugin-66b5774985-dprc5\" (UID: \"06a79981-dcc1-457b-bf83-200a86c08d63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" Jul 01 19:19:56.818995 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.818955 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/06a79981-dcc1-457b-bf83-200a86c08d63-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dprc5\" (UID: \"06a79981-dcc1-457b-bf83-200a86c08d63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" Jul 01 19:19:56.821389 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:56.821363 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/06a79981-dcc1-457b-bf83-200a86c08d63-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dprc5\" (UID: \"06a79981-dcc1-457b-bf83-200a86c08d63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" Jul 01 19:19:57.108290 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:57.108200 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" Jul 01 19:19:57.236915 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:57.236883 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5"] Jul 01 19:19:57.241696 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:19:57.241668 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06a79981_dcc1_457b_bf83_200a86c08d63.slice/crio-181bfef3e15005ff4b03f12fe1a28e9ded4a713777748450f0ac439c4b3997a0 WatchSource:0}: Error finding container 181bfef3e15005ff4b03f12fe1a28e9ded4a713777748450f0ac439c4b3997a0: Status 404 returned error can't find the container with id 181bfef3e15005ff4b03f12fe1a28e9ded4a713777748450f0ac439c4b3997a0 Jul 01 19:19:58.232479 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:19:58.232445 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" event={"ID":"06a79981-dcc1-457b-bf83-200a86c08d63","Type":"ContainerStarted","Data":"181bfef3e15005ff4b03f12fe1a28e9ded4a713777748450f0ac439c4b3997a0"} Jul 01 19:20:23.358537 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:23.358491 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" event={"ID":"06a79981-dcc1-457b-bf83-200a86c08d63","Type":"ContainerStarted","Data":"674726ee8256e9905167d1454ca8618aa755f6d44a3df0d3fa1591ab532f7502"} Jul 01 19:20:23.376870 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:23.376819 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dprc5" podStartSLOduration=2.229934088 podStartE2EDuration="27.376803345s" totalImagesPullingTime="25.146869257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:20:23.375658165 +0000 UTC m=+526.603832726" watchObservedRunningTime="2026-07-01 19:20:23.376803345 +0000 UTC m=+526.604977899" Jul 01 19:20:35.530654 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.530578 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx"] Jul 01 19:20:35.553426 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.553390 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx"] Jul 01 19:20:35.553579 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.553501 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.555809 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.555785 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-z96xm\"" Jul 01 19:20:35.583957 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.583919 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.584153 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.583964 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65cfda7c-a0b0-4b33-8141-464dc843aac6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.584153 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.583988 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.584153 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.584034 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.584153 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.584055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.584153 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.584070 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65cfda7c-a0b0-4b33-8141-464dc843aac6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.584153 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.584092 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65cfda7c-a0b0-4b33-8141-464dc843aac6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.584153 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.584148 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.584382 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.584165 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzw64\" (UniqueName: \"kubernetes.io/projected/65cfda7c-a0b0-4b33-8141-464dc843aac6-kube-api-access-xzw64\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.685491 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.685456 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65cfda7c-a0b0-4b33-8141-464dc843aac6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.685491 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.685495 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.685747 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.685530 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.685747 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.685552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.685747 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.685570 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65cfda7c-a0b0-4b33-8141-464dc843aac6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.685747 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.685590 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65cfda7c-a0b0-4b33-8141-464dc843aac6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.685747 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.685626 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.685747 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.685642 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzw64\" (UniqueName: \"kubernetes.io/projected/65cfda7c-a0b0-4b33-8141-464dc843aac6-kube-api-access-xzw64\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.685747 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.685691 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.686088 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.685923 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.686088 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.685992 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.686250 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.686090 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.686250 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.686168 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.686391 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.686375 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65cfda7c-a0b0-4b33-8141-464dc843aac6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.688035 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.688013 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65cfda7c-a0b0-4b33-8141-464dc843aac6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.688294 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.688274 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65cfda7c-a0b0-4b33-8141-464dc843aac6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.693450 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.693427 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65cfda7c-a0b0-4b33-8141-464dc843aac6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.693528 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.693477 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzw64\" (UniqueName: \"kubernetes.io/projected/65cfda7c-a0b0-4b33-8141-464dc843aac6-kube-api-access-xzw64\") pod \"maas-default-gateway-openshift-default-56654dcf7-m7bjx\" (UID: \"65cfda7c-a0b0-4b33-8141-464dc843aac6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.864415 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.864325 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:35.996772 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:35.996743 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx"] Jul 01 19:20:35.999206 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:20:35.999170 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65cfda7c_a0b0_4b33_8141_464dc843aac6.slice/crio-2241e170758ff4452cc9763052c57e70865ee98cd0d6b0665992f3765d1c063c WatchSource:0}: Error finding container 2241e170758ff4452cc9763052c57e70865ee98cd0d6b0665992f3765d1c063c: Status 404 returned error can't find the container with id 2241e170758ff4452cc9763052c57e70865ee98cd0d6b0665992f3765d1c063c Jul 01 19:20:36.001316 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:36.001284 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 19:20:36.001392 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:36.001355 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 19:20:36.001392 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:36.001384 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 19:20:36.416165 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:36.416122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" event={"ID":"65cfda7c-a0b0-4b33-8141-464dc843aac6","Type":"ContainerStarted","Data":"71839418c4673ef608198d8726355836195cac93bf76e092c15b754b0ceabcc8"} Jul 01 19:20:36.416374 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:36.416183 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" event={"ID":"65cfda7c-a0b0-4b33-8141-464dc843aac6","Type":"ContainerStarted","Data":"2241e170758ff4452cc9763052c57e70865ee98cd0d6b0665992f3765d1c063c"} Jul 01 19:20:36.436519 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:36.436474 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" podStartSLOduration=1.436455403 podStartE2EDuration="1.436455403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:20:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:20:36.434730787 +0000 UTC m=+539.662905359" watchObservedRunningTime="2026-07-01 19:20:36.436455403 +0000 UTC m=+539.664629958" Jul 01 19:20:36.865242 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:36.865200 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:37.870165 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:37.870135 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:38.424703 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:38.424668 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:38.425758 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:38.425735 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:20:38.560712 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:38.560680 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-t9hfk"] Jul 01 19:20:38.563107 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:38.563085 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t9hfk" Jul 01 19:20:38.566246 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:38.566228 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-prbkh\"" Jul 01 19:20:38.573210 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:38.573188 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t9hfk"] Jul 01 19:20:38.608109 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:38.608080 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdq8m\" (UniqueName: \"kubernetes.io/projected/36323cdd-4984-4645-b007-7380369f7674-kube-api-access-kdq8m\") pod \"authorino-75d556bd77-t9hfk\" (UID: \"36323cdd-4984-4645-b007-7380369f7674\") " pod="kuadrant-system/authorino-75d556bd77-t9hfk" Jul 01 19:20:38.708931 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:38.708896 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdq8m\" (UniqueName: \"kubernetes.io/projected/36323cdd-4984-4645-b007-7380369f7674-kube-api-access-kdq8m\") pod \"authorino-75d556bd77-t9hfk\" (UID: \"36323cdd-4984-4645-b007-7380369f7674\") " pod="kuadrant-system/authorino-75d556bd77-t9hfk" Jul 01 19:20:38.716435 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:38.716406 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdq8m\" (UniqueName: \"kubernetes.io/projected/36323cdd-4984-4645-b007-7380369f7674-kube-api-access-kdq8m\") pod \"authorino-75d556bd77-t9hfk\" (UID: \"36323cdd-4984-4645-b007-7380369f7674\") " pod="kuadrant-system/authorino-75d556bd77-t9hfk" Jul 01 19:20:38.872334 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:38.872298 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t9hfk" Jul 01 19:20:39.001212 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:39.000981 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t9hfk"] Jul 01 19:20:39.003843 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:20:39.003820 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36323cdd_4984_4645_b007_7380369f7674.slice/crio-769d4fe4e7f99c92fe328c4bc5f17bb4b2512962d9cd6ee452f9dfd2bf874f07 WatchSource:0}: Error finding container 769d4fe4e7f99c92fe328c4bc5f17bb4b2512962d9cd6ee452f9dfd2bf874f07: Status 404 returned error can't find the container with id 769d4fe4e7f99c92fe328c4bc5f17bb4b2512962d9cd6ee452f9dfd2bf874f07 Jul 01 19:20:39.429408 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:39.429328 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t9hfk" event={"ID":"36323cdd-4984-4645-b007-7380369f7674","Type":"ContainerStarted","Data":"769d4fe4e7f99c92fe328c4bc5f17bb4b2512962d9cd6ee452f9dfd2bf874f07"} Jul 01 19:20:42.445088 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:42.444991 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t9hfk" event={"ID":"36323cdd-4984-4645-b007-7380369f7674","Type":"ContainerStarted","Data":"eb4f3f38a7c3c40ed8faf23d7ac6ad6eecfe54829883bf8b569aa79f01177fce"} Jul 01 19:20:42.460856 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:20:42.460801 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-t9hfk" podStartSLOduration=1.281435878 podStartE2EDuration="4.460778578s" totalImagesPullingTime="3.1793427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:20:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:20:42.460428404 +0000 UTC m=+545.688602963" watchObservedRunningTime="2026-07-01 19:20:42.460778578 +0000 UTC m=+545.688953140" Jul 01 19:21:35.095993 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.095957 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 19:21:35.098642 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.098626 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 19:21:35.101100 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.101070 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 01 19:21:35.101100 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.101078 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 01 19:21:35.102804 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.102778 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 01 19:21:35.102925 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.102909 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-x5m2f\"" Jul 01 19:21:35.109308 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.109286 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 19:21:35.221283 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.221249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m52zm\" (UniqueName: \"kubernetes.io/projected/5cfe0d59-c382-48a0-b620-ba93a98bc046-kube-api-access-m52zm\") pod \"maas-keycloak-0\" (UID: \"5cfe0d59-c382-48a0-b620-ba93a98bc046\") " pod="keycloak-system/maas-keycloak-0" Jul 01 19:21:35.321850 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.321795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m52zm\" (UniqueName: \"kubernetes.io/projected/5cfe0d59-c382-48a0-b620-ba93a98bc046-kube-api-access-m52zm\") pod \"maas-keycloak-0\" (UID: \"5cfe0d59-c382-48a0-b620-ba93a98bc046\") " pod="keycloak-system/maas-keycloak-0" Jul 01 19:21:35.331351 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.331328 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m52zm\" (UniqueName: \"kubernetes.io/projected/5cfe0d59-c382-48a0-b620-ba93a98bc046-kube-api-access-m52zm\") pod \"maas-keycloak-0\" (UID: \"5cfe0d59-c382-48a0-b620-ba93a98bc046\") " pod="keycloak-system/maas-keycloak-0" Jul 01 19:21:35.408283 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.408193 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 19:21:35.550599 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.550577 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 19:21:35.552912 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:21:35.552886 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cfe0d59_c382_48a0_b620_ba93a98bc046.slice/crio-36804c69ea3001da12be44864cf90ff5087018ec5e44557de8a8ca4271e7c6f6 WatchSource:0}: Error finding container 36804c69ea3001da12be44864cf90ff5087018ec5e44557de8a8ca4271e7c6f6: Status 404 returned error can't find the container with id 36804c69ea3001da12be44864cf90ff5087018ec5e44557de8a8ca4271e7c6f6 Jul 01 19:21:35.669483 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:35.669393 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"5cfe0d59-c382-48a0-b620-ba93a98bc046","Type":"ContainerStarted","Data":"36804c69ea3001da12be44864cf90ff5087018ec5e44557de8a8ca4271e7c6f6"} Jul 01 19:21:37.202883 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:37.202853 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmnn_f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6/ovn-acl-logging/0.log" Jul 01 19:21:37.204222 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:37.204198 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmnn_f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6/ovn-acl-logging/0.log" Jul 01 19:21:40.356475 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:40.356453 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 01 19:21:40.698264 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:40.698155 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"5cfe0d59-c382-48a0-b620-ba93a98bc046","Type":"ContainerStarted","Data":"9f918209fef4f413d5e25fd132a93e75f5c6c1836fd3d3306f03e919e45aa3b8"} Jul 01 19:21:40.715708 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:40.715646 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.915971658 podStartE2EDuration="5.715624856s" totalImagesPullingTime="4.799653198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:21:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:40.714904343 +0000 UTC m=+603.943078899" watchObservedRunningTime="2026-07-01 19:21:40.715624856 +0000 UTC m=+603.943799412" Jul 01 19:21:41.408999 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:41.408950 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 01 19:21:41.411004 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:41.410949 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:42.409185 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:42.409139 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:43.409679 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:43.409635 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:44.409626 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:44.409583 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:45.409481 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:45.408736 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 01 19:21:45.410565 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:45.410527 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:46.409160 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:46.409085 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:47.410469 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:47.410422 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:48.409153 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:48.409077 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:49.409530 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:49.409476 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:50.409016 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:50.408959 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:51.409136 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:51.409067 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:52.409453 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:52.409414 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:53.409455 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:53.409401 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.27:9000/health/started\": dial tcp 10.134.0.27:9000: connect: connection refused" Jul 01 19:21:54.521697 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:54.521646 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 01 19:21:54.537792 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:21:54.537737 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:22:04.528463 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:04.528374 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 01 19:22:07.656381 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:07.656338 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t9hfk"] Jul 01 19:22:07.656878 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:07.656549 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-t9hfk" podUID="36323cdd-4984-4645-b007-7380369f7674" containerName="authorino" containerID="cri-o://eb4f3f38a7c3c40ed8faf23d7ac6ad6eecfe54829883bf8b569aa79f01177fce" gracePeriod=30 Jul 01 19:22:07.853952 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:07.853917 2577 generic.go:356] "Generic (PLEG): container finished" podID="36323cdd-4984-4645-b007-7380369f7674" containerID="eb4f3f38a7c3c40ed8faf23d7ac6ad6eecfe54829883bf8b569aa79f01177fce" exitCode=0 Jul 01 19:22:07.854139 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:07.854000 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t9hfk" event={"ID":"36323cdd-4984-4645-b007-7380369f7674","Type":"ContainerDied","Data":"eb4f3f38a7c3c40ed8faf23d7ac6ad6eecfe54829883bf8b569aa79f01177fce"} Jul 01 19:22:07.899459 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:07.899435 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t9hfk" Jul 01 19:22:07.952138 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:07.952097 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdq8m\" (UniqueName: \"kubernetes.io/projected/36323cdd-4984-4645-b007-7380369f7674-kube-api-access-kdq8m\") pod \"36323cdd-4984-4645-b007-7380369f7674\" (UID: \"36323cdd-4984-4645-b007-7380369f7674\") " Jul 01 19:22:07.954211 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:07.954190 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36323cdd-4984-4645-b007-7380369f7674-kube-api-access-kdq8m" (OuterVolumeSpecName: "kube-api-access-kdq8m") pod "36323cdd-4984-4645-b007-7380369f7674" (UID: "36323cdd-4984-4645-b007-7380369f7674"). InnerVolumeSpecName "kube-api-access-kdq8m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:22:08.052778 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:08.052741 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdq8m\" (UniqueName: \"kubernetes.io/projected/36323cdd-4984-4645-b007-7380369f7674-kube-api-access-kdq8m\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:22:08.859323 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:08.859295 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t9hfk" Jul 01 19:22:08.859323 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:08.859304 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t9hfk" event={"ID":"36323cdd-4984-4645-b007-7380369f7674","Type":"ContainerDied","Data":"769d4fe4e7f99c92fe328c4bc5f17bb4b2512962d9cd6ee452f9dfd2bf874f07"} Jul 01 19:22:08.859821 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:08.859356 2577 scope.go:117] "RemoveContainer" containerID="eb4f3f38a7c3c40ed8faf23d7ac6ad6eecfe54829883bf8b569aa79f01177fce" Jul 01 19:22:08.879293 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:08.879268 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t9hfk"] Jul 01 19:22:08.881067 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:08.881042 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t9hfk"] Jul 01 19:22:09.252331 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:09.252293 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36323cdd-4984-4645-b007-7380369f7674" path="/var/lib/kubelet/pods/36323cdd-4984-4645-b007-7380369f7674/volumes" Jul 01 19:22:11.969036 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:11.968996 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d96bf5b49-w98hr"] Jul 01 19:22:11.969544 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:11.969509 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36323cdd-4984-4645-b007-7380369f7674" containerName="authorino" Jul 01 19:22:11.969544 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:11.969528 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="36323cdd-4984-4645-b007-7380369f7674" containerName="authorino" Jul 01 19:22:11.969993 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:11.969611 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="36323cdd-4984-4645-b007-7380369f7674" containerName="authorino" Jul 01 19:22:11.972479 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:11.972459 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d96bf5b49-w98hr" Jul 01 19:22:11.974544 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:11.974515 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 01 19:22:11.974677 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:11.974545 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 01 19:22:11.974677 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:11.974654 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vlljv\"" Jul 01 19:22:11.982214 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:11.982190 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d96bf5b49-w98hr"] Jul 01 19:22:12.087414 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.087363 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxwvm\" (UniqueName: \"kubernetes.io/projected/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea-kube-api-access-kxwvm\") pod \"maas-controller-7d96bf5b49-w98hr\" (UID: \"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea\") " pod="opendatahub/maas-controller-7d96bf5b49-w98hr" Jul 01 19:22:12.087635 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.087493 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea-webhook-cert\") pod \"maas-controller-7d96bf5b49-w98hr\" (UID: \"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea\") " pod="opendatahub/maas-controller-7d96bf5b49-w98hr" Jul 01 19:22:12.147620 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.147587 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75c7f89dff-pnlrv"] Jul 01 19:22:12.149893 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.149878 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75c7f89dff-pnlrv" Jul 01 19:22:12.161487 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.161463 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75c7f89dff-pnlrv"] Jul 01 19:22:12.188889 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.188854 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea-webhook-cert\") pod \"maas-controller-7d96bf5b49-w98hr\" (UID: \"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea\") " pod="opendatahub/maas-controller-7d96bf5b49-w98hr" Jul 01 19:22:12.189054 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.188913 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxwvm\" (UniqueName: \"kubernetes.io/projected/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea-kube-api-access-kxwvm\") pod \"maas-controller-7d96bf5b49-w98hr\" (UID: \"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea\") " pod="opendatahub/maas-controller-7d96bf5b49-w98hr" Jul 01 19:22:12.191339 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.191320 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea-webhook-cert\") pod \"maas-controller-7d96bf5b49-w98hr\" (UID: \"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea\") " pod="opendatahub/maas-controller-7d96bf5b49-w98hr" Jul 01 19:22:12.198162 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.198136 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxwvm\" (UniqueName: \"kubernetes.io/projected/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea-kube-api-access-kxwvm\") pod \"maas-controller-7d96bf5b49-w98hr\" (UID: \"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea\") " pod="opendatahub/maas-controller-7d96bf5b49-w98hr" Jul 01 19:22:12.284669 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.284571 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d96bf5b49-w98hr" Jul 01 19:22:12.289825 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.289806 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ffcec722-3b99-4d4c-a145-72ec64ebdac8-webhook-cert\") pod \"maas-controller-75c7f89dff-pnlrv\" (UID: \"ffcec722-3b99-4d4c-a145-72ec64ebdac8\") " pod="opendatahub/maas-controller-75c7f89dff-pnlrv" Jul 01 19:22:12.289923 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.289839 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqn8l\" (UniqueName: \"kubernetes.io/projected/ffcec722-3b99-4d4c-a145-72ec64ebdac8-kube-api-access-xqn8l\") pod \"maas-controller-75c7f89dff-pnlrv\" (UID: \"ffcec722-3b99-4d4c-a145-72ec64ebdac8\") " pod="opendatahub/maas-controller-75c7f89dff-pnlrv" Jul 01 19:22:12.391490 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.391440 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ffcec722-3b99-4d4c-a145-72ec64ebdac8-webhook-cert\") pod \"maas-controller-75c7f89dff-pnlrv\" (UID: \"ffcec722-3b99-4d4c-a145-72ec64ebdac8\") " pod="opendatahub/maas-controller-75c7f89dff-pnlrv" Jul 01 19:22:12.391673 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.391517 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqn8l\" (UniqueName: \"kubernetes.io/projected/ffcec722-3b99-4d4c-a145-72ec64ebdac8-kube-api-access-xqn8l\") pod \"maas-controller-75c7f89dff-pnlrv\" (UID: \"ffcec722-3b99-4d4c-a145-72ec64ebdac8\") " pod="opendatahub/maas-controller-75c7f89dff-pnlrv" Jul 01 19:22:12.394189 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.394160 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ffcec722-3b99-4d4c-a145-72ec64ebdac8-webhook-cert\") pod \"maas-controller-75c7f89dff-pnlrv\" (UID: \"ffcec722-3b99-4d4c-a145-72ec64ebdac8\") " pod="opendatahub/maas-controller-75c7f89dff-pnlrv" Jul 01 19:22:12.400104 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.400080 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqn8l\" (UniqueName: \"kubernetes.io/projected/ffcec722-3b99-4d4c-a145-72ec64ebdac8-kube-api-access-xqn8l\") pod \"maas-controller-75c7f89dff-pnlrv\" (UID: \"ffcec722-3b99-4d4c-a145-72ec64ebdac8\") " pod="opendatahub/maas-controller-75c7f89dff-pnlrv" Jul 01 19:22:12.419844 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.419822 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d96bf5b49-w98hr"] Jul 01 19:22:12.421884 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:22:12.421856 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed1e6fb_116b_4a60_930e_b58a56f6a7ea.slice/crio-5a61cb4618b79b67adeea2278df2aba20fee725d5d9898170482e281940f1940 WatchSource:0}: Error finding container 5a61cb4618b79b67adeea2278df2aba20fee725d5d9898170482e281940f1940: Status 404 returned error can't find the container with id 5a61cb4618b79b67adeea2278df2aba20fee725d5d9898170482e281940f1940 Jul 01 19:22:12.460958 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.460914 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75c7f89dff-pnlrv" Jul 01 19:22:12.592461 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.592434 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75c7f89dff-pnlrv"] Jul 01 19:22:12.594764 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:22:12.594740 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffcec722_3b99_4d4c_a145_72ec64ebdac8.slice/crio-d763d78afd5c0afaaee3bc3b6a66b20bc4a0a46653f9f37dff57a0fe3579fb4d WatchSource:0}: Error finding container d763d78afd5c0afaaee3bc3b6a66b20bc4a0a46653f9f37dff57a0fe3579fb4d: Status 404 returned error can't find the container with id d763d78afd5c0afaaee3bc3b6a66b20bc4a0a46653f9f37dff57a0fe3579fb4d Jul 01 19:22:12.879992 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.879902 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75c7f89dff-pnlrv" event={"ID":"ffcec722-3b99-4d4c-a145-72ec64ebdac8","Type":"ContainerStarted","Data":"d763d78afd5c0afaaee3bc3b6a66b20bc4a0a46653f9f37dff57a0fe3579fb4d"} Jul 01 19:22:12.881099 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:12.881075 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d96bf5b49-w98hr" event={"ID":"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea","Type":"ContainerStarted","Data":"5a61cb4618b79b67adeea2278df2aba20fee725d5d9898170482e281940f1940"} Jul 01 19:22:15.901224 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:15.901122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d96bf5b49-w98hr" event={"ID":"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea","Type":"ContainerStarted","Data":"264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5"} Jul 01 19:22:15.901224 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:15.901190 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d96bf5b49-w98hr" Jul 01 19:22:15.903043 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:15.902544 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75c7f89dff-pnlrv" event={"ID":"ffcec722-3b99-4d4c-a145-72ec64ebdac8","Type":"ContainerStarted","Data":"d959017f6138b5bf79970b7674c65fb1584c0ed46217c1810a98c02658518446"} Jul 01 19:22:15.903043 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:15.902591 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75c7f89dff-pnlrv" Jul 01 19:22:15.923835 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:15.923785 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d96bf5b49-w98hr" podStartSLOduration=2.131158973 podStartE2EDuration="4.923770412s" totalImagesPullingTime="2.792611439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:15.920207437 +0000 UTC m=+639.148381992" watchObservedRunningTime="2026-07-01 19:22:15.923770412 +0000 UTC m=+639.151944967" Jul 01 19:22:15.942841 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:15.942791 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75c7f89dff-pnlrv" podStartSLOduration=1.327719203 podStartE2EDuration="3.942774814s" totalImagesPullingTime="2.615055611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:15.941550153 +0000 UTC m=+639.169724720" watchObservedRunningTime="2026-07-01 19:22:15.942774814 +0000 UTC m=+639.170949370" Jul 01 19:22:26.913307 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:26.913270 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d96bf5b49-w98hr" Jul 01 19:22:26.913881 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:26.913481 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75c7f89dff-pnlrv" Jul 01 19:22:26.971942 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:26.971902 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d96bf5b49-w98hr"] Jul 01 19:22:26.972251 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:26.972217 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7d96bf5b49-w98hr" podUID="2ed1e6fb-116b-4a60-930e-b58a56f6a7ea" containerName="manager" containerID="cri-o://264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5" gracePeriod=30 Jul 01 19:22:27.220194 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.220172 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d96bf5b49-w98hr" Jul 01 19:22:27.331562 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.331512 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea-webhook-cert\") pod \"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea\" (UID: \"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea\") " Jul 01 19:22:27.331754 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.331671 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxwvm\" (UniqueName: \"kubernetes.io/projected/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea-kube-api-access-kxwvm\") pod \"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea\" (UID: \"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea\") " Jul 01 19:22:27.333711 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.333678 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2ed1e6fb-116b-4a60-930e-b58a56f6a7ea" (UID: "2ed1e6fb-116b-4a60-930e-b58a56f6a7ea"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:22:27.333813 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.333759 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea-kube-api-access-kxwvm" (OuterVolumeSpecName: "kube-api-access-kxwvm") pod "2ed1e6fb-116b-4a60-930e-b58a56f6a7ea" (UID: "2ed1e6fb-116b-4a60-930e-b58a56f6a7ea"). InnerVolumeSpecName "kube-api-access-kxwvm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:22:27.432982 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.432939 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxwvm\" (UniqueName: \"kubernetes.io/projected/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea-kube-api-access-kxwvm\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:22:27.432982 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.432975 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea-webhook-cert\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:22:27.955349 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.955312 2577 generic.go:356] "Generic (PLEG): container finished" podID="2ed1e6fb-116b-4a60-930e-b58a56f6a7ea" containerID="264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5" exitCode=0 Jul 01 19:22:27.955731 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.955354 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d96bf5b49-w98hr" Jul 01 19:22:27.955731 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.955359 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d96bf5b49-w98hr" event={"ID":"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea","Type":"ContainerDied","Data":"264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5"} Jul 01 19:22:27.955731 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.955416 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d96bf5b49-w98hr" event={"ID":"2ed1e6fb-116b-4a60-930e-b58a56f6a7ea","Type":"ContainerDied","Data":"5a61cb4618b79b67adeea2278df2aba20fee725d5d9898170482e281940f1940"} Jul 01 19:22:27.955731 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.955440 2577 scope.go:117] "RemoveContainer" containerID="264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5" Jul 01 19:22:27.964066 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.964047 2577 scope.go:117] "RemoveContainer" containerID="264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5" Jul 01 19:22:27.964348 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:22:27.964328 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5\": container with ID starting with 264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5 not found: ID does not exist" containerID="264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5" Jul 01 19:22:27.964420 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.964363 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5"} err="failed to get container status \"264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5\": rpc error: code = NotFound desc = could not find container \"264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5\": container with ID starting with 264fdf5b3b54656e3071be7574f688d6b5b249f77db862d2431a0c13141acbd5 not found: ID does not exist" Jul 01 19:22:27.976370 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.976346 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d96bf5b49-w98hr"] Jul 01 19:22:27.979708 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:27.979685 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d96bf5b49-w98hr"] Jul 01 19:22:29.252219 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:29.252188 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ed1e6fb-116b-4a60-930e-b58a56f6a7ea" path="/var/lib/kubelet/pods/2ed1e6fb-116b-4a60-930e-b58a56f6a7ea/volumes" Jul 01 19:22:49.288806 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.288769 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6b8657d754-dj6nn"] Jul 01 19:22:49.289392 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.289373 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ed1e6fb-116b-4a60-930e-b58a56f6a7ea" containerName="manager" Jul 01 19:22:49.289454 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.289397 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ed1e6fb-116b-4a60-930e-b58a56f6a7ea" containerName="manager" Jul 01 19:22:49.289524 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.289513 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ed1e6fb-116b-4a60-930e-b58a56f6a7ea" containerName="manager" Jul 01 19:22:49.310023 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.309989 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6b8657d754-dj6nn"] Jul 01 19:22:49.310199 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.310103 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:22:49.312238 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.312213 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 01 19:22:49.312369 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.312288 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-ds9zg\"" Jul 01 19:22:49.423327 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.423287 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fd780f1-28e2-4728-9b47-05072e6fcb5e-maas-api-tls\") pod \"maas-api-6b8657d754-dj6nn\" (UID: \"9fd780f1-28e2-4728-9b47-05072e6fcb5e\") " pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:22:49.423327 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.423328 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l59pb\" (UniqueName: \"kubernetes.io/projected/9fd780f1-28e2-4728-9b47-05072e6fcb5e-kube-api-access-l59pb\") pod \"maas-api-6b8657d754-dj6nn\" (UID: \"9fd780f1-28e2-4728-9b47-05072e6fcb5e\") " pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:22:49.423550 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.423354 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd780f1-28e2-4728-9b47-05072e6fcb5e-openshift-service-ca\") pod \"maas-api-6b8657d754-dj6nn\" (UID: \"9fd780f1-28e2-4728-9b47-05072e6fcb5e\") " pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:22:49.524328 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.524286 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd780f1-28e2-4728-9b47-05072e6fcb5e-openshift-service-ca\") pod \"maas-api-6b8657d754-dj6nn\" (UID: \"9fd780f1-28e2-4728-9b47-05072e6fcb5e\") " pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:22:49.524503 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.524420 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fd780f1-28e2-4728-9b47-05072e6fcb5e-maas-api-tls\") pod \"maas-api-6b8657d754-dj6nn\" (UID: \"9fd780f1-28e2-4728-9b47-05072e6fcb5e\") " pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:22:49.524503 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.524447 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l59pb\" (UniqueName: \"kubernetes.io/projected/9fd780f1-28e2-4728-9b47-05072e6fcb5e-kube-api-access-l59pb\") pod \"maas-api-6b8657d754-dj6nn\" (UID: \"9fd780f1-28e2-4728-9b47-05072e6fcb5e\") " pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:22:49.524974 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.524954 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd780f1-28e2-4728-9b47-05072e6fcb5e-openshift-service-ca\") pod \"maas-api-6b8657d754-dj6nn\" (UID: \"9fd780f1-28e2-4728-9b47-05072e6fcb5e\") " pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:22:49.526968 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.526939 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fd780f1-28e2-4728-9b47-05072e6fcb5e-maas-api-tls\") pod \"maas-api-6b8657d754-dj6nn\" (UID: \"9fd780f1-28e2-4728-9b47-05072e6fcb5e\") " pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:22:49.532905 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.532878 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l59pb\" (UniqueName: \"kubernetes.io/projected/9fd780f1-28e2-4728-9b47-05072e6fcb5e-kube-api-access-l59pb\") pod \"maas-api-6b8657d754-dj6nn\" (UID: \"9fd780f1-28e2-4728-9b47-05072e6fcb5e\") " pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:22:49.621873 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.621787 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:22:49.790180 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.790156 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6b8657d754-dj6nn"] Jul 01 19:22:49.793099 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:22:49.793066 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd780f1_28e2_4728_9b47_05072e6fcb5e.slice/crio-478ccc275aaa95245a1de7be66fcf60a39e0beb4b8d2bc97a1b25859ae48b06d WatchSource:0}: Error finding container 478ccc275aaa95245a1de7be66fcf60a39e0beb4b8d2bc97a1b25859ae48b06d: Status 404 returned error can't find the container with id 478ccc275aaa95245a1de7be66fcf60a39e0beb4b8d2bc97a1b25859ae48b06d Jul 01 19:22:49.794883 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.794862 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 19:22:49.910230 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.910149 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf"] Jul 01 19:22:49.913245 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.913227 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" Jul 01 19:22:49.915444 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.915422 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zlk4p\"" Jul 01 19:22:49.915555 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.915429 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 01 19:22:49.921655 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.921631 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf"] Jul 01 19:22:49.969728 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.969698 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-65g9n"] Jul 01 19:22:49.972098 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.972067 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:22:49.981321 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:49.981281 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-65g9n"] Jul 01 19:22:50.029098 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.029060 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cd2b4bd-cb72-48c9-92aa-fa46672971ae-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-65g9n\" (UID: \"8cd2b4bd-cb72-48c9-92aa-fa46672971ae\") " pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:22:50.029098 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.029099 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm9fw\" (UniqueName: \"kubernetes.io/projected/28119b69-6ae5-4e8c-897c-f99cf8cd1829-kube-api-access-xm9fw\") pod \"payload-pre-processing-59f49fd4f6-gktnf\" (UID: \"28119b69-6ae5-4e8c-897c-f99cf8cd1829\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" Jul 01 19:22:50.029379 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.029299 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28119b69-6ae5-4e8c-897c-f99cf8cd1829-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-gktnf\" (UID: \"28119b69-6ae5-4e8c-897c-f99cf8cd1829\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" Jul 01 19:22:50.029379 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.029340 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lthr6\" (UniqueName: \"kubernetes.io/projected/8cd2b4bd-cb72-48c9-92aa-fa46672971ae-kube-api-access-lthr6\") pod \"payload-processing-7bbcb654f4-65g9n\" (UID: \"8cd2b4bd-cb72-48c9-92aa-fa46672971ae\") " pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:22:50.054386 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.054350 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6b8657d754-dj6nn" event={"ID":"9fd780f1-28e2-4728-9b47-05072e6fcb5e","Type":"ContainerStarted","Data":"478ccc275aaa95245a1de7be66fcf60a39e0beb4b8d2bc97a1b25859ae48b06d"} Jul 01 19:22:50.130602 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.130564 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28119b69-6ae5-4e8c-897c-f99cf8cd1829-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-gktnf\" (UID: \"28119b69-6ae5-4e8c-897c-f99cf8cd1829\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" Jul 01 19:22:50.130784 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.130610 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lthr6\" (UniqueName: \"kubernetes.io/projected/8cd2b4bd-cb72-48c9-92aa-fa46672971ae-kube-api-access-lthr6\") pod \"payload-processing-7bbcb654f4-65g9n\" (UID: \"8cd2b4bd-cb72-48c9-92aa-fa46672971ae\") " pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:22:50.130784 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.130655 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cd2b4bd-cb72-48c9-92aa-fa46672971ae-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-65g9n\" (UID: \"8cd2b4bd-cb72-48c9-92aa-fa46672971ae\") " pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:22:50.130784 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.130686 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9fw\" (UniqueName: \"kubernetes.io/projected/28119b69-6ae5-4e8c-897c-f99cf8cd1829-kube-api-access-xm9fw\") pod \"payload-pre-processing-59f49fd4f6-gktnf\" (UID: \"28119b69-6ae5-4e8c-897c-f99cf8cd1829\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" Jul 01 19:22:50.131513 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.131483 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28119b69-6ae5-4e8c-897c-f99cf8cd1829-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-gktnf\" (UID: \"28119b69-6ae5-4e8c-897c-f99cf8cd1829\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" Jul 01 19:22:50.131636 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.131557 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cd2b4bd-cb72-48c9-92aa-fa46672971ae-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-65g9n\" (UID: \"8cd2b4bd-cb72-48c9-92aa-fa46672971ae\") " pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:22:50.139629 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.139596 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm9fw\" (UniqueName: \"kubernetes.io/projected/28119b69-6ae5-4e8c-897c-f99cf8cd1829-kube-api-access-xm9fw\") pod \"payload-pre-processing-59f49fd4f6-gktnf\" (UID: \"28119b69-6ae5-4e8c-897c-f99cf8cd1829\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" Jul 01 19:22:50.140030 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.140004 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lthr6\" (UniqueName: \"kubernetes.io/projected/8cd2b4bd-cb72-48c9-92aa-fa46672971ae-kube-api-access-lthr6\") pod \"payload-processing-7bbcb654f4-65g9n\" (UID: \"8cd2b4bd-cb72-48c9-92aa-fa46672971ae\") " pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:22:50.225139 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.225080 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" Jul 01 19:22:50.288446 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.288413 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:22:50.363272 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.363243 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf"] Jul 01 19:22:50.364669 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:22:50.364628 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28119b69_6ae5_4e8c_897c_f99cf8cd1829.slice/crio-9b10f16f2259f38459a7300c8e04886a89994cd41fe371610cb02fbc2d61b7d0 WatchSource:0}: Error finding container 9b10f16f2259f38459a7300c8e04886a89994cd41fe371610cb02fbc2d61b7d0: Status 404 returned error can't find the container with id 9b10f16f2259f38459a7300c8e04886a89994cd41fe371610cb02fbc2d61b7d0 Jul 01 19:22:50.434764 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:50.434733 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-65g9n"] Jul 01 19:22:50.437530 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:22:50.437504 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cd2b4bd_cb72_48c9_92aa_fa46672971ae.slice/crio-a975b87656e944ab7b72b56ec1a9baba1bae2411b4530a81d72346d75b880028 WatchSource:0}: Error finding container a975b87656e944ab7b72b56ec1a9baba1bae2411b4530a81d72346d75b880028: Status 404 returned error can't find the container with id a975b87656e944ab7b72b56ec1a9baba1bae2411b4530a81d72346d75b880028 Jul 01 19:22:51.069596 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:51.069509 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" event={"ID":"28119b69-6ae5-4e8c-897c-f99cf8cd1829","Type":"ContainerStarted","Data":"9b10f16f2259f38459a7300c8e04886a89994cd41fe371610cb02fbc2d61b7d0"} Jul 01 19:22:51.073026 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:51.072990 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" event={"ID":"8cd2b4bd-cb72-48c9-92aa-fa46672971ae","Type":"ContainerStarted","Data":"a975b87656e944ab7b72b56ec1a9baba1bae2411b4530a81d72346d75b880028"} Jul 01 19:22:54.097407 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:54.097366 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" event={"ID":"28119b69-6ae5-4e8c-897c-f99cf8cd1829","Type":"ContainerStarted","Data":"296a32aaec00934d84e7219c778c903294b72469dc924ba0f8b6c3be25d210e6"} Jul 01 19:22:54.097861 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:54.097470 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" Jul 01 19:22:54.099022 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:54.098997 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6b8657d754-dj6nn" event={"ID":"9fd780f1-28e2-4728-9b47-05072e6fcb5e","Type":"ContainerStarted","Data":"c8dbb5775c5204c6fb1a4709269281ab2667ea0e9871adc9fa83e9d60d9f73b6"} Jul 01 19:22:54.099138 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:54.099062 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:22:54.100483 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:54.100457 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" event={"ID":"8cd2b4bd-cb72-48c9-92aa-fa46672971ae","Type":"ContainerStarted","Data":"84654f2d301b8a2c0d98cd1ced5a04e27588bb7f92c46a077fd02ca336d1764e"} Jul 01 19:22:54.100610 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:54.100585 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:22:54.114150 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:54.114080 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" podStartSLOduration=1.7920898539999999 podStartE2EDuration="5.114064997s" totalImagesPullingTime="3.321975143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:54.112179793 +0000 UTC m=+677.340354346" watchObservedRunningTime="2026-07-01 19:22:54.114064997 +0000 UTC m=+677.342239551" Jul 01 19:22:54.127281 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:54.127229 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" podStartSLOduration=1.887839155 podStartE2EDuration="5.127212259s" totalImagesPullingTime="3.239373104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:54.126192748 +0000 UTC m=+677.354367305" watchObservedRunningTime="2026-07-01 19:22:54.127212259 +0000 UTC m=+677.355386814" Jul 01 19:22:54.152092 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:54.152012 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6b8657d754-dj6nn" podStartSLOduration=1.3243121389999999 podStartE2EDuration="5.151987903s" totalImagesPullingTime="3.827675764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:54.151598023 +0000 UTC m=+677.379772578" watchObservedRunningTime="2026-07-01 19:22:54.151987903 +0000 UTC m=+677.380162459" Jul 01 19:22:58.181396 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:22:58.181362 2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65cfda7c_a0b0_4b33_8141_464dc843aac6.slice/crio-71839418c4673ef608198d8726355836195cac93bf76e092c15b754b0ceabcc8.scope\": RecentStats: unable to find data in memory cache]" Jul 01 19:22:59.130093 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:59.130056 2577 generic.go:356] "Generic (PLEG): container finished" podID="65cfda7c-a0b0-4b33-8141-464dc843aac6" containerID="71839418c4673ef608198d8726355836195cac93bf76e092c15b754b0ceabcc8" exitCode=0 Jul 01 19:22:59.130302 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:59.130107 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" event={"ID":"65cfda7c-a0b0-4b33-8141-464dc843aac6","Type":"ContainerDied","Data":"71839418c4673ef608198d8726355836195cac93bf76e092c15b754b0ceabcc8"} Jul 01 19:22:59.130512 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:59.130493 2577 scope.go:117] "RemoveContainer" containerID="71839418c4673ef608198d8726355836195cac93bf76e092c15b754b0ceabcc8" Jul 01 19:22:59.131476 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:59.131447 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 19:22:59.131540 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:59.131513 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 19:22:59.131578 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:22:59.131545 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 19:23:00.111589 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:00.111556 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6b8657d754-dj6nn" Jul 01 19:23:00.137858 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:00.137820 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" event={"ID":"65cfda7c-a0b0-4b33-8141-464dc843aac6","Type":"ContainerStarted","Data":"84b57331f267b53f0ce2bdffa8ca7b2021d6c6b32b0d7413da641056a92bd34e"} Jul 01 19:23:00.864619 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:00.864582 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:23:00.864806 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:00.864785 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m7bjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 01 19:23:00.864854 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:00.864834 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" podUID="65cfda7c-a0b0-4b33-8141-464dc843aac6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 01 19:23:01.566832 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:01.566800 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 19:23:01.567306 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:01.567016 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" containerID="cri-o://9f918209fef4f413d5e25fd132a93e75f5c6c1836fd3d3306f03e919e45aa3b8" gracePeriod=30 Jul 01 19:23:01.864935 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:01.864844 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m7bjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 01 19:23:01.864935 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:01.864916 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" podUID="65cfda7c-a0b0-4b33-8141-464dc843aac6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 01 19:23:02.865472 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:02.865433 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m7bjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 01 19:23:02.866009 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:02.865485 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" podUID="65cfda7c-a0b0-4b33-8141-464dc843aac6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 01 19:23:03.177580 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:03.177485 2577 generic.go:356] "Generic (PLEG): container finished" podID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerID="9f918209fef4f413d5e25fd132a93e75f5c6c1836fd3d3306f03e919e45aa3b8" exitCode=143 Jul 01 19:23:03.177580 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:03.177517 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"5cfe0d59-c382-48a0-b620-ba93a98bc046","Type":"ContainerDied","Data":"9f918209fef4f413d5e25fd132a93e75f5c6c1836fd3d3306f03e919e45aa3b8"} Jul 01 19:23:03.268386 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:03.268353 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:03.383067 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:03.383030 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m52zm\" (UniqueName: \"kubernetes.io/projected/5cfe0d59-c382-48a0-b620-ba93a98bc046-kube-api-access-m52zm\") pod \"5cfe0d59-c382-48a0-b620-ba93a98bc046\" (UID: \"5cfe0d59-c382-48a0-b620-ba93a98bc046\") " Jul 01 19:23:03.385386 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:03.385356 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cfe0d59-c382-48a0-b620-ba93a98bc046-kube-api-access-m52zm" (OuterVolumeSpecName: "kube-api-access-m52zm") pod "5cfe0d59-c382-48a0-b620-ba93a98bc046" (UID: "5cfe0d59-c382-48a0-b620-ba93a98bc046"). InnerVolumeSpecName "kube-api-access-m52zm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:23:03.484192 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:03.484154 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m52zm\" (UniqueName: \"kubernetes.io/projected/5cfe0d59-c382-48a0-b620-ba93a98bc046-kube-api-access-m52zm\") on node \"ip-10-0-142-98.ec2.internal\" DevicePath \"\"" Jul 01 19:23:03.865068 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:03.864973 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m7bjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 01 19:23:03.865068 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:03.865038 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" podUID="65cfda7c-a0b0-4b33-8141-464dc843aac6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 01 19:23:04.183380 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.183301 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"5cfe0d59-c382-48a0-b620-ba93a98bc046","Type":"ContainerDied","Data":"36804c69ea3001da12be44864cf90ff5087018ec5e44557de8a8ca4271e7c6f6"} Jul 01 19:23:04.183779 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.183312 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:04.183779 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.183386 2577 scope.go:117] "RemoveContainer" containerID="9f918209fef4f413d5e25fd132a93e75f5c6c1836fd3d3306f03e919e45aa3b8" Jul 01 19:23:04.206343 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.206317 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 19:23:04.210561 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.210535 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 19:23:04.232684 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.232650 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 19:23:04.233057 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.233040 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" Jul 01 19:23:04.233159 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.233061 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" Jul 01 19:23:04.233228 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.233212 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" containerName="keycloak" Jul 01 19:23:04.236141 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.236104 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:04.238386 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.238359 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 01 19:23:04.238386 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.238359 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 01 19:23:04.238534 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.238404 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-x5m2f\"" Jul 01 19:23:04.238534 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.238408 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 01 19:23:04.238534 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.238489 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jul 01 19:23:04.244532 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.244511 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 19:23:04.391306 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.391263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/b57d36cb-1e67-4ebf-936d-7a3319ac05e2-test-realms\") pod \"maas-keycloak-0\" (UID: \"b57d36cb-1e67-4ebf-936d-7a3319ac05e2\") " pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:04.391306 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.391306 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-878m4\" (UniqueName: \"kubernetes.io/projected/b57d36cb-1e67-4ebf-936d-7a3319ac05e2-kube-api-access-878m4\") pod \"maas-keycloak-0\" (UID: \"b57d36cb-1e67-4ebf-936d-7a3319ac05e2\") " pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:04.491969 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.491930 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/b57d36cb-1e67-4ebf-936d-7a3319ac05e2-test-realms\") pod \"maas-keycloak-0\" (UID: \"b57d36cb-1e67-4ebf-936d-7a3319ac05e2\") " pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:04.491969 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.491964 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878m4\" (UniqueName: \"kubernetes.io/projected/b57d36cb-1e67-4ebf-936d-7a3319ac05e2-kube-api-access-878m4\") pod \"maas-keycloak-0\" (UID: \"b57d36cb-1e67-4ebf-936d-7a3319ac05e2\") " pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:04.492677 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.492655 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/b57d36cb-1e67-4ebf-936d-7a3319ac05e2-test-realms\") pod \"maas-keycloak-0\" (UID: \"b57d36cb-1e67-4ebf-936d-7a3319ac05e2\") " pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:04.500018 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.499998 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-878m4\" (UniqueName: \"kubernetes.io/projected/b57d36cb-1e67-4ebf-936d-7a3319ac05e2-kube-api-access-878m4\") pod \"maas-keycloak-0\" (UID: \"b57d36cb-1e67-4ebf-936d-7a3319ac05e2\") " pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:04.547827 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.547790 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:04.686381 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.686356 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 19:23:04.688994 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:23:04.688966 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb57d36cb_1e67_4ebf_936d_7a3319ac05e2.slice/crio-9021531cab9ea596565ebd714623853bde9b7e8aaa4afcf4882ca08e39aa0e96 WatchSource:0}: Error finding container 9021531cab9ea596565ebd714623853bde9b7e8aaa4afcf4882ca08e39aa0e96: Status 404 returned error can't find the container with id 9021531cab9ea596565ebd714623853bde9b7e8aaa4afcf4882ca08e39aa0e96 Jul 01 19:23:04.868863 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.868784 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:23:04.869159 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.869140 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:23:04.869942 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:04.869923 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m7bjx" Jul 01 19:23:05.108001 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:05.107975 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:23:05.108180 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:05.108054 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-gktnf" Jul 01 19:23:05.189715 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:05.189681 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"b57d36cb-1e67-4ebf-936d-7a3319ac05e2","Type":"ContainerStarted","Data":"9021531cab9ea596565ebd714623853bde9b7e8aaa4afcf4882ca08e39aa0e96"} Jul 01 19:23:05.253270 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:05.253232 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cfe0d59-c382-48a0-b620-ba93a98bc046" path="/var/lib/kubelet/pods/5cfe0d59-c382-48a0-b620-ba93a98bc046/volumes" Jul 01 19:23:06.197400 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:06.197344 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"b57d36cb-1e67-4ebf-936d-7a3319ac05e2","Type":"ContainerStarted","Data":"3bdee7a13bb70b527e0e8311d7ddbe02ce443ce3a6af19fd7ac664f2ded57254"} Jul 01 19:23:06.217481 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:06.217407 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.745440101 podStartE2EDuration="2.217382244s" totalImagesPullingTime="471.942143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:23:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:06.215324889 +0000 UTC m=+689.443499470" watchObservedRunningTime="2026-07-01 19:23:06.217382244 +0000 UTC m=+689.445556800" Jul 01 19:23:06.548951 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:06.548899 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:06.550975 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:06.550932 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:07.548879 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:07.548829 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:08.549031 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:08.548974 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:09.548956 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:09.548907 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:10.549809 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:10.549757 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:11.548654 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:11.548601 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:12.549234 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:12.549181 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:13.548685 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:13.548640 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:14.548626 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:14.548569 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:14.549176 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:14.548889 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:15.548994 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:15.548949 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:16.548604 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:16.548565 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:17.548317 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:17.548271 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:18.548886 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:18.548842 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 19:23:19.670224 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:19.670184 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:19.688798 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:19.688740 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="b57d36cb-1e67-4ebf-936d-7a3319ac05e2" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:23:29.676730 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:29.676634 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 01 19:23:55.889669 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:55.889639 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn"] Jul 01 19:23:55.894484 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:55.894467 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:55.896919 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:55.896894 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 01 19:23:55.896919 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:55.896897 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mf94x\"" Jul 01 19:23:55.897229 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:55.897213 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 01 19:23:55.897323 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:55.897217 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 01 19:23:55.904014 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:55.903994 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn"] Jul 01 19:23:55.980882 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:55.980845 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7vh4\" (UniqueName: \"kubernetes.io/projected/3794f55d-3818-4aa0-8619-638115075b5d-kube-api-access-l7vh4\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:55.981070 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:55.980903 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3794f55d-3818-4aa0-8619-638115075b5d-dshm\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:55.981070 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:55.981001 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3794f55d-3818-4aa0-8619-638115075b5d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:55.981070 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:55.981035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3794f55d-3818-4aa0-8619-638115075b5d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:55.981226 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:55.981075 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3794f55d-3818-4aa0-8619-638115075b5d-home\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:56.081802 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.081771 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7vh4\" (UniqueName: \"kubernetes.io/projected/3794f55d-3818-4aa0-8619-638115075b5d-kube-api-access-l7vh4\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:56.082002 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.081827 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3794f55d-3818-4aa0-8619-638115075b5d-dshm\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:56.082002 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.081858 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3794f55d-3818-4aa0-8619-638115075b5d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:56.082002 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.081875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3794f55d-3818-4aa0-8619-638115075b5d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:56.082002 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.081908 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3794f55d-3818-4aa0-8619-638115075b5d-home\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:56.082288 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.082269 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3794f55d-3818-4aa0-8619-638115075b5d-home\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:56.082657 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.082336 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3794f55d-3818-4aa0-8619-638115075b5d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:56.084279 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.084257 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3794f55d-3818-4aa0-8619-638115075b5d-dshm\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:56.084381 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.084366 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3794f55d-3818-4aa0-8619-638115075b5d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:56.090569 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.090544 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7vh4\" (UniqueName: \"kubernetes.io/projected/3794f55d-3818-4aa0-8619-638115075b5d-kube-api-access-l7vh4\") pod \"e2e-distinct-2-simulated-kserve-74f954f784-p87wn\" (UID: \"3794f55d-3818-4aa0-8619-638115075b5d\") " pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:56.205290 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.205258 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:23:56.340439 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.340415 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn"] Jul 01 19:23:56.343797 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:23:56.343770 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3794f55d_3818_4aa0_8619_638115075b5d.slice/crio-57226f33684b8184d6e01833ad2809f06d5dba2f7f17900247e86c32641ee0ae WatchSource:0}: Error finding container 57226f33684b8184d6e01833ad2809f06d5dba2f7f17900247e86c32641ee0ae: Status 404 returned error can't find the container with id 57226f33684b8184d6e01833ad2809f06d5dba2f7f17900247e86c32641ee0ae Jul 01 19:23:56.477291 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:56.477207 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" event={"ID":"3794f55d-3818-4aa0-8619-638115075b5d","Type":"ContainerStarted","Data":"57226f33684b8184d6e01833ad2809f06d5dba2f7f17900247e86c32641ee0ae"} Jul 01 19:23:58.488765 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:58.488728 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" event={"ID":"3794f55d-3818-4aa0-8619-638115075b5d","Type":"ContainerStarted","Data":"9de5f8d0824e0042fafb5a3af235827332e0b0f9218c7bd0e09ed27033543307"} Jul 01 19:23:58.507694 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:23:58.507649 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" podStartSLOduration=2.346815058 podStartE2EDuration="3.507631956s" totalImagesPullingTime="1.160816898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:58.506144973 +0000 UTC m=+741.734319521" watchObservedRunningTime="2026-07-01 19:23:58.507631956 +0000 UTC m=+741.735806510" Jul 01 19:24:06.206007 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:06.205967 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:24:06.206568 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:06.206039 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:24:06.218205 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:06.218181 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:24:06.535403 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:06.535310 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-74f954f784-p87wn" Jul 01 19:24:28.681616 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.681579 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr"] Jul 01 19:24:28.687878 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.687847 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.689922 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.689899 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 01 19:24:28.696776 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.696757 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr"] Jul 01 19:24:28.789878 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.789841 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06940784-d26a-4c94-b9c3-03c7c0481ef3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.789878 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.789881 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06940784-d26a-4c94-b9c3-03c7c0481ef3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.790090 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.789976 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt44c\" (UniqueName: \"kubernetes.io/projected/06940784-d26a-4c94-b9c3-03c7c0481ef3-kube-api-access-qt44c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.790090 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.790001 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06940784-d26a-4c94-b9c3-03c7c0481ef3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.790090 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.790021 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06940784-d26a-4c94-b9c3-03c7c0481ef3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.891347 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.891306 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06940784-d26a-4c94-b9c3-03c7c0481ef3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.891347 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.891349 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06940784-d26a-4c94-b9c3-03c7c0481ef3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.891567 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.891413 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt44c\" (UniqueName: \"kubernetes.io/projected/06940784-d26a-4c94-b9c3-03c7c0481ef3-kube-api-access-qt44c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.891567 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.891438 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06940784-d26a-4c94-b9c3-03c7c0481ef3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.891567 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.891466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06940784-d26a-4c94-b9c3-03c7c0481ef3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.891785 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.891763 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06940784-d26a-4c94-b9c3-03c7c0481ef3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.891884 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.891862 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06940784-d26a-4c94-b9c3-03c7c0481ef3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.893695 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.893672 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06940784-d26a-4c94-b9c3-03c7c0481ef3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.893846 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.893831 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06940784-d26a-4c94-b9c3-03c7c0481ef3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:28.899216 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:28.899191 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt44c\" (UniqueName: \"kubernetes.io/projected/06940784-d26a-4c94-b9c3-03c7c0481ef3-kube-api-access-qt44c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr\" (UID: \"06940784-d26a-4c94-b9c3-03c7c0481ef3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:29.000069 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:29.000025 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:29.137292 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:29.137267 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr"] Jul 01 19:24:29.139547 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:24:29.139517 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06940784_d26a_4c94_b9c3_03c7c0481ef3.slice/crio-b3d2b21987c55835f4a611ac9b45ebabec2aef54a939ac7e82d8d44d76608b93 WatchSource:0}: Error finding container b3d2b21987c55835f4a611ac9b45ebabec2aef54a939ac7e82d8d44d76608b93: Status 404 returned error can't find the container with id b3d2b21987c55835f4a611ac9b45ebabec2aef54a939ac7e82d8d44d76608b93 Jul 01 19:24:29.629127 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:29.629015 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" event={"ID":"06940784-d26a-4c94-b9c3-03c7c0481ef3","Type":"ContainerStarted","Data":"db5f33075bc8fe0d159195aa7c358931bab857bcf7c8b793363378c14968e061"} Jul 01 19:24:29.629127 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:29.629073 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" event={"ID":"06940784-d26a-4c94-b9c3-03c7c0481ef3","Type":"ContainerStarted","Data":"b3d2b21987c55835f4a611ac9b45ebabec2aef54a939ac7e82d8d44d76608b93"} Jul 01 19:24:29.647153 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:29.647088 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" podStartSLOduration=1.460820284 podStartE2EDuration="1.647070572s" totalImagesPullingTime="186.250288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:29.645920917 +0000 UTC m=+772.874095470" watchObservedRunningTime="2026-07-01 19:24:29.647070572 +0000 UTC m=+772.875245129" Jul 01 19:24:39.000367 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:39.000322 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:39.000861 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:39.000412 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:39.012997 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:39.012973 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:24:39.691396 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:24:39.691363 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr" Jul 01 19:26:20.138327 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:20.138294 2577 generic.go:356] "Generic (PLEG): container finished" podID="8cd2b4bd-cb72-48c9-92aa-fa46672971ae" containerID="84654f2d301b8a2c0d98cd1ced5a04e27588bb7f92c46a077fd02ca336d1764e" exitCode=2 Jul 01 19:26:20.138327 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:20.138363 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" event={"ID":"8cd2b4bd-cb72-48c9-92aa-fa46672971ae","Type":"ContainerDied","Data":"84654f2d301b8a2c0d98cd1ced5a04e27588bb7f92c46a077fd02ca336d1764e"} Jul 01 19:26:20.138862 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:20.138764 2577 scope.go:117] "RemoveContainer" containerID="84654f2d301b8a2c0d98cd1ced5a04e27588bb7f92c46a077fd02ca336d1764e" Jul 01 19:26:21.144928 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:21.144892 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" event={"ID":"8cd2b4bd-cb72-48c9-92aa-fa46672971ae","Type":"ContainerStarted","Data":"1e669c3b5e7409b626785c57eab2c5d1043e99f02c3fe8a31177e6df4f547ce6"} Jul 01 19:26:21.145383 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:21.145104 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:26:32.152221 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:32.152142 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:26:37.233706 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:37.233677 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmnn_f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6/ovn-acl-logging/0.log" Jul 01 19:26:37.235705 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:37.235684 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmnn_f5f8c521-cbd7-4873-8b71-9ad27d4bb3f6/ovn-acl-logging/0.log" Jul 01 19:26:52.150475 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:52.150422 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" podUID="8cd2b4bd-cb72-48c9-92aa-fa46672971ae" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.32:9004: connect: connection refused" Jul 01 19:26:52.292788 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:52.292754 2577 generic.go:356] "Generic (PLEG): container finished" podID="8cd2b4bd-cb72-48c9-92aa-fa46672971ae" containerID="1e669c3b5e7409b626785c57eab2c5d1043e99f02c3fe8a31177e6df4f547ce6" exitCode=2 Jul 01 19:26:52.292960 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:52.292820 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" event={"ID":"8cd2b4bd-cb72-48c9-92aa-fa46672971ae","Type":"ContainerDied","Data":"1e669c3b5e7409b626785c57eab2c5d1043e99f02c3fe8a31177e6df4f547ce6"} Jul 01 19:26:52.292960 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:52.292882 2577 scope.go:117] "RemoveContainer" containerID="84654f2d301b8a2c0d98cd1ced5a04e27588bb7f92c46a077fd02ca336d1764e" Jul 01 19:26:52.293276 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:52.293249 2577 scope.go:117] "RemoveContainer" containerID="1e669c3b5e7409b626785c57eab2c5d1043e99f02c3fe8a31177e6df4f547ce6" Jul 01 19:26:52.293536 ip-10-0-142-98 kubenswrapper[2577]: E0701 19:26:52.293514 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7bbcb654f4-65g9n_openshift-ingress(8cd2b4bd-cb72-48c9-92aa-fa46672971ae)\"" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" podUID="8cd2b4bd-cb72-48c9-92aa-fa46672971ae" Jul 01 19:26:57.110277 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:57.110249 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6b8657d754-dj6nn_9fd780f1-28e2-4728-9b47-05072e6fcb5e/maas-api/0.log" Jul 01 19:26:57.225952 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:57.225920 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-75c7f89dff-pnlrv_ffcec722-3b99-4d4c-a145-72ec64ebdac8/manager/0.log" Jul 01 19:26:57.560962 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:57.560928 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85b6b744df-fjhzb_c2437657-336f-463e-8d52-8adda08ff3cf/copy-manifests/0.log" Jul 01 19:26:57.574458 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:57.574429 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85b6b744df-fjhzb_c2437657-336f-463e-8d52-8adda08ff3cf/manager/0.log" Jul 01 19:26:59.516919 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:26:59.516875 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dprc5_06a79981-dcc1-457b-bf83-200a86c08d63/kuadrant-console-plugin/0.log" Jul 01 19:27:00.359966 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:00.359933 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf49hl42_829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f/istio-proxy/0.log" Jul 01 19:27:00.591469 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:00.591438 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-658989848c-6bsd6_4dcfecef-d3d5-48ff-af17-4a4cabb63c28/kube-auth-proxy/0.log" Jul 01 19:27:00.827316 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:00.827280 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m7bjx_65cfda7c-a0b0-4b33-8141-464dc843aac6/istio-proxy/1.log" Jul 01 19:27:00.942186 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:00.942154 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-gktnf_28119b69-6ae5-4e8c-897c-f99cf8cd1829/payload-pre-processing/0.log" Jul 01 19:27:01.052872 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:01.052827 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-65g9n_8cd2b4bd-cb72-48c9-92aa-fa46672971ae/payload-processing/1.log" Jul 01 19:27:01.176832 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:01.176740 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b759854c7-dwz6b_d0c6e2e7-7f8a-4911-a5e1-1933759bb182/router/0.log" Jul 01 19:27:01.518787 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:01.518757 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-74f954f784-p87wn_3794f55d-3818-4aa0-8619-638115075b5d/main/0.log" Jul 01 19:27:01.858769 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:01.858668 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-66d649s2vwr_06940784-d26a-4c94-b9c3-03c7c0481ef3/main/0.log" Jul 01 19:27:02.150061 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:02.149969 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:27:02.150418 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:02.150403 2577 scope.go:117] "RemoveContainer" containerID="1e669c3b5e7409b626785c57eab2c5d1043e99f02c3fe8a31177e6df4f547ce6" Jul 01 19:27:02.347149 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:02.347090 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" event={"ID":"8cd2b4bd-cb72-48c9-92aa-fa46672971ae","Type":"ContainerStarted","Data":"e60edb5a9dcc7322176b9416286191305ed3ec35969fb6385a3a7f7e17a7f237"} Jul 01 19:27:02.347423 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:02.347396 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:27:09.817235 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:09.817204 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-66xvf_943fd8e6-36e9-45d0-a60f-78d295d95d9f/global-pull-secret-syncer/0.log" Jul 01 19:27:09.962087 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:09.962052 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cqtzm_05fffc12-7ff5-4ae0-9f3d-85267bb7eaef/konnectivity-agent/0.log" Jul 01 19:27:10.069884 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:10.069792 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-98.ec2.internal_8247c4c3a47d291912a129baeb7becc3/haproxy/0.log" Jul 01 19:27:13.354182 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:13.354149 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-65g9n" Jul 01 19:27:15.057869 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:15.057842 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dprc5_06a79981-dcc1-457b-bf83-200a86c08d63/kuadrant-console-plugin/0.log" Jul 01 19:27:17.173679 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.173590 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-vxf6s_6dbf42f7-f386-43e5-bfa9-8b9d2a4117f0/cluster-monitoring-operator/0.log" Jul 01 19:27:17.202685 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.202659 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-r9rll_801580e2-ea20-4b49-bd1b-95248917be1c/kube-state-metrics/0.log" Jul 01 19:27:17.233294 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.233267 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-r9rll_801580e2-ea20-4b49-bd1b-95248917be1c/kube-rbac-proxy-main/0.log" Jul 01 19:27:17.265041 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.265000 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-r9rll_801580e2-ea20-4b49-bd1b-95248917be1c/kube-rbac-proxy-self/0.log" Jul 01 19:27:17.300184 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.300157 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-644dbb586b-s6r75_cc9f4385-41d2-40ff-96ce-d1bbd2d1b0a7/metrics-server/0.log" Jul 01 19:27:17.333066 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.333036 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-jjn9q_2b4b2bad-d026-423e-bbed-fa190bdafa3b/monitoring-plugin/0.log" Jul 01 19:27:17.383336 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.383309 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qjn5l_193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb/node-exporter/0.log" Jul 01 19:27:17.405918 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.405887 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qjn5l_193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb/kube-rbac-proxy/0.log" Jul 01 19:27:17.433852 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.433779 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qjn5l_193cc7a8-c0bb-49b5-8bfb-19b19e8c48eb/init-textfile/0.log" Jul 01 19:27:17.886475 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.886444 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f1662f3-0094-44fc-aead-a1159a6856c7/prometheus/0.log" Jul 01 19:27:17.914770 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.914745 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f1662f3-0094-44fc-aead-a1159a6856c7/config-reloader/0.log" Jul 01 19:27:17.950926 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.950901 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f1662f3-0094-44fc-aead-a1159a6856c7/thanos-sidecar/0.log" Jul 01 19:27:17.976580 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:17.976555 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f1662f3-0094-44fc-aead-a1159a6856c7/kube-rbac-proxy-web/0.log" Jul 01 19:27:18.002542 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:18.002514 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f1662f3-0094-44fc-aead-a1159a6856c7/kube-rbac-proxy/0.log" Jul 01 19:27:18.033128 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:18.033100 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f1662f3-0094-44fc-aead-a1159a6856c7/kube-rbac-proxy-thanos/0.log" Jul 01 19:27:18.061344 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:18.061317 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f1662f3-0094-44fc-aead-a1159a6856c7/init-config-reloader/0.log" Jul 01 19:27:18.097831 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:18.097799 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-md7rc_ef6ec9c8-c755-4724-97ba-128f7f3f8e8d/prometheus-operator/0.log" Jul 01 19:27:18.126976 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:18.126922 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-md7rc_ef6ec9c8-c755-4724-97ba-128f7f3f8e8d/kube-rbac-proxy/0.log" Jul 01 19:27:20.217174 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.217140 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx"] Jul 01 19:27:20.220506 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.220490 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.223063 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.223041 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tj2n4\"/\"kube-root-ca.crt\"" Jul 01 19:27:20.223213 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.223041 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tj2n4\"/\"default-dockercfg-bh6ct\"" Jul 01 19:27:20.223495 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.223478 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tj2n4\"/\"openshift-service-ca.crt\"" Jul 01 19:27:20.227920 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.227896 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx"] Jul 01 19:27:20.308014 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.307959 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f8247ec2-75f3-4ec1-97a7-b25098e481a7-podres\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.308237 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.308031 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f8247ec2-75f3-4ec1-97a7-b25098e481a7-proc\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.308237 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.308076 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8247ec2-75f3-4ec1-97a7-b25098e481a7-sys\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.308237 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.308096 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8247ec2-75f3-4ec1-97a7-b25098e481a7-lib-modules\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.308337 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.308239 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgkvm\" (UniqueName: \"kubernetes.io/projected/f8247ec2-75f3-4ec1-97a7-b25098e481a7-kube-api-access-xgkvm\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.409431 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.409386 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgkvm\" (UniqueName: \"kubernetes.io/projected/f8247ec2-75f3-4ec1-97a7-b25098e481a7-kube-api-access-xgkvm\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.409644 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.409465 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f8247ec2-75f3-4ec1-97a7-b25098e481a7-podres\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.409644 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.409500 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f8247ec2-75f3-4ec1-97a7-b25098e481a7-proc\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.409644 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.409541 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8247ec2-75f3-4ec1-97a7-b25098e481a7-sys\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.409644 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.409566 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8247ec2-75f3-4ec1-97a7-b25098e481a7-lib-modules\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.409644 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.409605 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f8247ec2-75f3-4ec1-97a7-b25098e481a7-proc\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.409644 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.409637 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f8247ec2-75f3-4ec1-97a7-b25098e481a7-podres\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.409860 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.409688 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8247ec2-75f3-4ec1-97a7-b25098e481a7-sys\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.409860 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.409715 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8247ec2-75f3-4ec1-97a7-b25098e481a7-lib-modules\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.417777 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.417747 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgkvm\" (UniqueName: \"kubernetes.io/projected/f8247ec2-75f3-4ec1-97a7-b25098e481a7-kube-api-access-xgkvm\") pod \"perf-node-gather-daemonset-t28gx\" (UID: \"f8247ec2-75f3-4ec1-97a7-b25098e481a7\") " pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.531841 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.531746 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:20.666137 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:20.666095 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx"] Jul 01 19:27:20.667810 ip-10-0-142-98 kubenswrapper[2577]: W0701 19:27:20.667775 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf8247ec2_75f3_4ec1_97a7_b25098e481a7.slice/crio-c096eedcfa8b25ce99d5f9177087a497f2cc8d2d3b78b3507dc9fc23149ef1e6 WatchSource:0}: Error finding container c096eedcfa8b25ce99d5f9177087a497f2cc8d2d3b78b3507dc9fc23149ef1e6: Status 404 returned error can't find the container with id c096eedcfa8b25ce99d5f9177087a497f2cc8d2d3b78b3507dc9fc23149ef1e6 Jul 01 19:27:21.437275 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:21.437236 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" event={"ID":"f8247ec2-75f3-4ec1-97a7-b25098e481a7","Type":"ContainerStarted","Data":"343e87d10c751e140a2f407ab1878a9dfc1e67f222dc52a0e8415fbd502e9f05"} Jul 01 19:27:21.437769 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:21.437292 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" event={"ID":"f8247ec2-75f3-4ec1-97a7-b25098e481a7","Type":"ContainerStarted","Data":"c096eedcfa8b25ce99d5f9177087a497f2cc8d2d3b78b3507dc9fc23149ef1e6"} Jul 01 19:27:21.437769 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:21.437377 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:21.455085 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:21.455032 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" podStartSLOduration=1.455016563 podStartE2EDuration="1.455016563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:27:21.454036487 +0000 UTC m=+944.682211041" watchObservedRunningTime="2026-07-01 19:27:21.455016563 +0000 UTC m=+944.683191157" Jul 01 19:27:21.456352 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:21.456332 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-69d9f679db-k8mbf_9cf3071f-e57f-4f32-9ee8-5f441663ad3c/volume-data-source-validator/0.log" Jul 01 19:27:22.617932 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:22.617903 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rcb2k_8855e158-eba7-4470-b0be-cce54da3d10b/dns/0.log" Jul 01 19:27:22.644169 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:22.644145 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rcb2k_8855e158-eba7-4470-b0be-cce54da3d10b/kube-rbac-proxy/0.log" Jul 01 19:27:22.671604 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:22.671573 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7tp8j_691b723a-898b-4dbc-81ee-6646ef35df0a/dns-node-resolver/0.log" Jul 01 19:27:23.350744 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:23.350714 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bmzq6_99b873f3-650c-4b66-acf7-5eb80ecb4c70/node-ca/0.log" Jul 01 19:27:24.365078 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:24.365047 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf49hl42_829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f/istio-proxy/0.log" Jul 01 19:27:24.451688 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:24.451662 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-658989848c-6bsd6_4dcfecef-d3d5-48ff-af17-4a4cabb63c28/kube-auth-proxy/0.log" Jul 01 19:27:24.514792 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:24.514763 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m7bjx_65cfda7c-a0b0-4b33-8141-464dc843aac6/istio-proxy/1.log" Jul 01 19:27:24.515054 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:24.515037 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m7bjx_65cfda7c-a0b0-4b33-8141-464dc843aac6/istio-proxy/0.log" Jul 01 19:27:24.537883 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:24.537859 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-gktnf_28119b69-6ae5-4e8c-897c-f99cf8cd1829/payload-pre-processing/0.log" Jul 01 19:27:24.570483 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:24.570452 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-65g9n_8cd2b4bd-cb72-48c9-92aa-fa46672971ae/payload-processing/2.log" Jul 01 19:27:24.570955 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:24.570934 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-65g9n_8cd2b4bd-cb72-48c9-92aa-fa46672971ae/payload-processing/1.log" Jul 01 19:27:24.594870 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:24.594833 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b759854c7-dwz6b_d0c6e2e7-7f8a-4911-a5e1-1933759bb182/router/0.log" Jul 01 19:27:25.252888 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:25.252861 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6q7sh_9ab889f3-cd15-4323-b69a-e8b1141710ec/serve-healthcheck-canary/0.log" Jul 01 19:27:25.959752 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:25.959722 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rtwv8_19a0267f-762f-4fb6-971d-f5988a038b5a/kube-rbac-proxy/0.log" Jul 01 19:27:25.991881 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:25.991851 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rtwv8_19a0267f-762f-4fb6-971d-f5988a038b5a/exporter/0.log" Jul 01 19:27:26.017799 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:26.017775 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rtwv8_19a0267f-762f-4fb6-971d-f5988a038b5a/extractor/0.log" Jul 01 19:27:27.456458 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:27.456430 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tj2n4/perf-node-gather-daemonset-t28gx" Jul 01 19:27:28.516595 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:28.516545 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6b8657d754-dj6nn_9fd780f1-28e2-4728-9b47-05072e6fcb5e/maas-api/0.log" Jul 01 19:27:28.563708 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:28.563674 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-75c7f89dff-pnlrv_ffcec722-3b99-4d4c-a145-72ec64ebdac8/manager/0.log" Jul 01 19:27:28.725380 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:28.725353 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85b6b744df-fjhzb_c2437657-336f-463e-8d52-8adda08ff3cf/manager/0.log" Jul 01 19:27:28.752693 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:28.752669 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85b6b744df-fjhzb_c2437657-336f-463e-8d52-8adda08ff3cf/copy-manifests/0.log" Jul 01 19:27:30.600626 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:30.600601 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6858c4896d-hlzqx_dbfc6a7c-66b5-4db8-b7a6-7ba8da9c9996/manager/0.log" Jul 01 19:27:30.734150 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:30.734100 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf49hl42_829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f/istio-proxy/0.log" Jul 01 19:27:30.808168 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:30.808140 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-658989848c-6bsd6_4dcfecef-d3d5-48ff-af17-4a4cabb63c28/kube-auth-proxy/0.log" Jul 01 19:27:30.843383 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:30.843359 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m7bjx_65cfda7c-a0b0-4b33-8141-464dc843aac6/istio-proxy/1.log" Jul 01 19:27:30.843629 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:30.843581 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m7bjx_65cfda7c-a0b0-4b33-8141-464dc843aac6/istio-proxy/0.log" Jul 01 19:27:30.853518 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:30.853431 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-gktnf_28119b69-6ae5-4e8c-897c-f99cf8cd1829/payload-pre-processing/0.log" Jul 01 19:27:30.869030 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:30.869007 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-65g9n_8cd2b4bd-cb72-48c9-92aa-fa46672971ae/payload-processing/2.log" Jul 01 19:27:30.869570 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:30.869547 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-65g9n_8cd2b4bd-cb72-48c9-92aa-fa46672971ae/payload-processing/1.log" Jul 01 19:27:30.884718 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:30.884697 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b759854c7-dwz6b_d0c6e2e7-7f8a-4911-a5e1-1933759bb182/router/0.log" Jul 01 19:27:32.113141 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:32.113050 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf49hl42_829721a6-f175-4f7b-b0d7-4d3f4b2e1a9f/istio-proxy/0.log" Jul 01 19:27:32.191539 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:32.191507 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-658989848c-6bsd6_4dcfecef-d3d5-48ff-af17-4a4cabb63c28/kube-auth-proxy/0.log" Jul 01 19:27:32.232837 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:32.232779 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m7bjx_65cfda7c-a0b0-4b33-8141-464dc843aac6/istio-proxy/1.log" Jul 01 19:27:32.233097 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:32.233075 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m7bjx_65cfda7c-a0b0-4b33-8141-464dc843aac6/istio-proxy/0.log" Jul 01 19:27:32.249255 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:32.249228 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-gktnf_28119b69-6ae5-4e8c-897c-f99cf8cd1829/payload-pre-processing/0.log" Jul 01 19:27:32.267174 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:32.267145 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-65g9n_8cd2b4bd-cb72-48c9-92aa-fa46672971ae/payload-processing/2.log" Jul 01 19:27:32.267679 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:32.267663 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-65g9n_8cd2b4bd-cb72-48c9-92aa-fa46672971ae/payload-processing/1.log" Jul 01 19:27:32.284733 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:32.284691 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b759854c7-dwz6b_d0c6e2e7-7f8a-4911-a5e1-1933759bb182/router/0.log" Jul 01 19:27:37.082842 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:37.082812 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-24dr2_409fb45f-4128-440a-98ec-58763de6e171/kube-multus/0.log" Jul 01 19:27:37.359952 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:37.359875 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xlfr9_771dc0a4-073a-4c63-85c5-e405dad55641/kube-multus-additional-cni-plugins/0.log" Jul 01 19:27:37.391637 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:37.391613 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xlfr9_771dc0a4-073a-4c63-85c5-e405dad55641/egress-router-binary-copy/0.log" Jul 01 19:27:37.417321 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:37.417295 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xlfr9_771dc0a4-073a-4c63-85c5-e405dad55641/cni-plugins/0.log" Jul 01 19:27:37.446416 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:37.446380 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xlfr9_771dc0a4-073a-4c63-85c5-e405dad55641/bond-cni-plugin/0.log" Jul 01 19:27:37.479676 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:37.479653 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xlfr9_771dc0a4-073a-4c63-85c5-e405dad55641/routeoverride-cni/0.log" Jul 01 19:27:37.503997 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:37.503971 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xlfr9_771dc0a4-073a-4c63-85c5-e405dad55641/whereabouts-cni-bincopy/0.log" Jul 01 19:27:37.533484 ip-10-0-142-98 kubenswrapper[2577]: I0701 19:27:37.533462 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xlfr9_771dc0a4-073a-4c63-85c5-e405dad55641/whereabouts-cni/0.log"