Sep 06 12:44:53.240427 ip-10-0-141-13 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 06 12:44:53.240436 ip-10-0-141-13 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 06 12:44:53.240443 ip-10-0-141-13 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 06 12:44:53.240584 ip-10-0-141-13 systemd[1]: Failed to start Kubernetes Kubelet. Sep 06 12:45:02.599563 ip-10-0-141-13 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 06 12:45:02.599575 ip-10-0-141-13 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 8c6432341ea0410e9e4d0ac0827fca26 -- Sep 06 12:46:30.126562 ip-10-0-141-13 systemd[1]: Starting Kubernetes Kubelet... Sep 06 12:46:30.567324 ip-10-0-141-13 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 06 12:46:30.567324 ip-10-0-141-13 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 06 12:46:30.567324 ip-10-0-141-13 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 06 12:46:30.567324 ip-10-0-141-13 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 06 12:46:30.567324 ip-10-0-141-13 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 06 12:46:30.569943 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.569855 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 06 12:46:30.574115 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574080 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 12:46:30.574115 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574111 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 12:46:30.574115 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574118 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574135 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574140 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574144 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574148 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574152 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574156 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574160 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574165 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574169 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574173 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574177 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574182 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574186 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574195 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574199 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574204 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574208 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574212 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574222 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 12:46:30.574329 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574227 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574233 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574239 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574243 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574247 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574252 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574275 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574280 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574285 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574289 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574294 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574298 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574302 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574306 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574311 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574315 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574320 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574324 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574328 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574338 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 12:46:30.574845 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574342 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574347 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574351 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574356 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574360 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574364 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574368 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574375 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574414 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574452 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574556 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574563 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574566 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574570 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574573 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574576 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574578 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574581 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574584 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 12:46:30.575437 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574586 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574590 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574593 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574595 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574598 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574601 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574604 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574607 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574610 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574618 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574621 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574623 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574626 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574630 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574633 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574636 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574638 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574641 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574644 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574646 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 12:46:30.575899 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574649 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574651 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574654 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574657 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574660 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574663 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574668 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574671 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574674 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574677 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574680 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574682 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574687 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574690 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574693 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574697 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574700 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.574702 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575451 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 12:46:30.576397 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575458 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575462 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575465 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575468 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575471 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575474 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575477 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575480 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575482 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575485 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575488 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575491 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575493 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575496 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575498 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575503 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575505 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575508 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575510 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575513 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575515 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 12:46:30.576850 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575518 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575520 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575522 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575525 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575527 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575530 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575532 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575535 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575538 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575540 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575543 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575545 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575548 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575550 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575553 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575555 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575558 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575560 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575563 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 12:46:30.577388 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575565 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575568 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575570 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575573 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575576 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575578 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575581 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575584 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575587 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575590 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575593 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575595 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575598 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575601 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575605 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575609 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575612 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575616 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575619 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 12:46:30.577902 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575621 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575624 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575627 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575630 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575633 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575635 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575638 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575641 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575644 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575646 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575649 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575651 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575654 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575656 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575659 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575662 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575665 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575667 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575670 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 12:46:30.578416 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575672 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575675 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575678 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575680 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575683 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575686 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575689 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575692 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575695 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575697 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575699 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575702 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575704 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575707 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575709 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575713 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575717 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575720 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575723 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 12:46:30.578878 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.575725 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576098 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576103 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576106 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576108 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576111 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576114 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576117 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576120 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576123 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576125 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576128 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576131 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576133 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576136 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576138 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576141 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576144 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576147 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576150 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 12:46:30.579353 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576152 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576155 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576157 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576160 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576164 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576168 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576173 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576177 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576180 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576183 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576186 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576189 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576192 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576194 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576197 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576199 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576202 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576205 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576207 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 12:46:30.579840 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576210 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576212 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576215 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576220 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576222 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576225 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576227 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576231 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576234 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576237 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576240 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576243 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576246 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576248 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576251 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576253 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576256 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576272 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576275 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576278 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 12:46:30.580335 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576280 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576283 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576285 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576288 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576291 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576294 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576297 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576299 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576303 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576306 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576308 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576311 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576313 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576316 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576318 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576322 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576325 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576327 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576330 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576332 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 12:46:30.580828 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576335 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576337 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576340 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576342 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576346 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576348 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576351 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576353 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576356 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576358 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576361 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576363 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576366 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576369 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576371 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576374 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576377 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576379 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576382 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576384 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 12:46:30.581346 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.576387 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576462 2580 flags.go:64] FLAG: --address="0.0.0.0" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576499 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576505 2580 flags.go:64] FLAG: --anonymous-auth="true" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576510 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576513 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576516 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576521 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576526 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576529 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576534 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576537 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576541 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576544 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576547 2580 flags.go:64] FLAG: --cgroup-root="" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576549 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576552 2580 flags.go:64] FLAG: --client-ca-file="" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576556 2580 flags.go:64] FLAG: --cloud-provider="external" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576559 2580 flags.go:64] FLAG: --cluster-dns="[]" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576563 2580 flags.go:64] FLAG: --cluster-domain="" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576565 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576568 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576572 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576575 2580 flags.go:64] FLAG: --container-log-max-files="5" Sep 06 12:46:30.581829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576579 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576581 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576584 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576587 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576590 2580 flags.go:64] FLAG: --contention-profiling="false" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576593 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576596 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576599 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576601 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576605 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576608 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576611 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576614 2580 flags.go:64] FLAG: --enable-load-reader="false" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576616 2580 flags.go:64] FLAG: --enable-server="true" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576619 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576624 2580 flags.go:64] FLAG: --event-burst="100" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576627 2580 flags.go:64] FLAG: --event-qps="50" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576630 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576634 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576637 2580 flags.go:64] FLAG: --eviction-hard="" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576641 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576644 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576647 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576650 2580 flags.go:64] FLAG: --eviction-soft="" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576653 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 06 12:46:30.582436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576656 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576659 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576662 2580 flags.go:64] FLAG: --experimental-mounter-path="" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576665 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576667 2580 flags.go:64] FLAG: --fail-swap-on="true" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576670 2580 flags.go:64] FLAG: --feature-gates="" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576674 2580 flags.go:64] FLAG: --file-check-frequency="20s" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576677 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576680 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576683 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576686 2580 flags.go:64] FLAG: --healthz-port="10248" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576689 2580 flags.go:64] FLAG: --help="false" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576692 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-141-13.ec2.internal" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576695 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576698 2580 flags.go:64] FLAG: --http-check-frequency="20s" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576701 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576704 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576707 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576710 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576713 2580 flags.go:64] FLAG: --image-service-endpoint="" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576716 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576718 2580 flags.go:64] FLAG: --kube-api-burst="100" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576721 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576724 2580 flags.go:64] FLAG: --kube-api-qps="50" Sep 06 12:46:30.583086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576727 2580 flags.go:64] FLAG: --kube-reserved="" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576731 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576734 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576737 2580 flags.go:64] FLAG: --kubelet-cgroups="" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576739 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576742 2580 flags.go:64] FLAG: --lock-file="" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576745 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576747 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576750 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576755 2580 flags.go:64] FLAG: --log-json-split-stream="false" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576759 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576761 2580 flags.go:64] FLAG: --log-text-split-stream="false" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576764 2580 flags.go:64] FLAG: --logging-format="text" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576767 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576770 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576773 2580 flags.go:64] FLAG: --manifest-url="" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576776 2580 flags.go:64] FLAG: --manifest-url-header="" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576780 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576783 2580 flags.go:64] FLAG: --max-open-files="1000000" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576787 2580 flags.go:64] FLAG: --max-pods="110" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576790 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576793 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576796 2580 flags.go:64] FLAG: --memory-manager-policy="None" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576799 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576801 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 06 12:46:30.583701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576804 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576807 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576814 2580 flags.go:64] FLAG: --node-status-max-images="50" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576817 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576820 2580 flags.go:64] FLAG: --oom-score-adj="-999" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576823 2580 flags.go:64] FLAG: --pod-cidr="" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576826 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576831 2580 flags.go:64] FLAG: --pod-manifest-path="" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576835 2580 flags.go:64] FLAG: --pod-max-pids="-1" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576838 2580 flags.go:64] FLAG: --pods-per-core="0" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576841 2580 flags.go:64] FLAG: --port="10250" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576844 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576846 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01d6f8b5c33445d91" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576850 2580 flags.go:64] FLAG: --qos-reserved="" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576852 2580 flags.go:64] FLAG: --read-only-port="10255" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576856 2580 flags.go:64] FLAG: --register-node="true" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576858 2580 flags.go:64] FLAG: --register-with-taints="" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576862 2580 flags.go:64] FLAG: --registry-burst="10" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576866 2580 flags.go:64] FLAG: --registry-qps="5" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576869 2580 flags.go:64] FLAG: --reserved-cpus="" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576871 2580 flags.go:64] FLAG: --reserved-memory="" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576875 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576878 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576881 2580 flags.go:64] FLAG: --rotate-certificates="false" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576883 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 06 12:46:30.584362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576886 2580 flags.go:64] FLAG: --runonce="false" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576890 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576893 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576896 2580 flags.go:64] FLAG: --seccomp-default="false" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576899 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576901 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576904 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576907 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576910 2580 flags.go:64] FLAG: --storage-driver-password="root" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576913 2580 flags.go:64] FLAG: --storage-driver-secure="false" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576916 2580 flags.go:64] FLAG: --storage-driver-table="stats" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576919 2580 flags.go:64] FLAG: --storage-driver-user="root" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576922 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576925 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576928 2580 flags.go:64] FLAG: --system-cgroups="" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576932 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576937 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576940 2580 flags.go:64] FLAG: --tls-cert-file="" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576943 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576947 2580 flags.go:64] FLAG: --tls-min-version="" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576950 2580 flags.go:64] FLAG: --tls-private-key-file="" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576953 2580 flags.go:64] FLAG: --topology-manager-policy="none" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576956 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576958 2580 flags.go:64] FLAG: --topology-manager-scope="container" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576961 2580 flags.go:64] FLAG: --v="2" Sep 06 12:46:30.584977 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576965 2580 flags.go:64] FLAG: --version="false" Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576969 2580 flags.go:64] FLAG: --vmodule="" Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576973 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.576976 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577093 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577096 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577100 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577103 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577109 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577113 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577116 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577118 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577121 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577123 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577126 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577128 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577131 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577133 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577136 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577139 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 12:46:30.585597 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577142 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577144 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577148 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577151 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577153 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577156 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577159 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577161 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577164 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577166 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577169 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577172 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577174 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577177 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577181 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577183 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577186 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577188 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577191 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 12:46:30.586077 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577194 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577199 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577203 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577205 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577208 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577210 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577213 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577216 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577218 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577221 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577223 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577226 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577229 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577233 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577236 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577240 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577243 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577246 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577248 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577251 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 12:46:30.586579 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577254 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577256 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577271 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577274 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577277 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577280 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577282 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577285 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577288 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577290 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577293 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577296 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577298 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577302 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577305 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577308 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577310 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577312 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577315 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577318 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 12:46:30.587076 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577320 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577323 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577325 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577328 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577330 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577333 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577335 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577339 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577341 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577344 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577347 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577349 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577351 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577354 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577357 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577359 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577362 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577364 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577367 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 12:46:30.587638 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577369 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577371 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577374 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577377 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.577380 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.578236 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.585627 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.585643 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.586501 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.586509 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586563 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586568 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586572 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586575 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586578 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 12:46:30.588101 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586581 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586584 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586587 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586590 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586593 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586596 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586599 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586601 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586604 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586607 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586609 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586612 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586615 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586618 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586621 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586623 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586626 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586629 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586632 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586635 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 12:46:30.588504 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586638 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586641 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586644 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586648 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586654 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586658 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586662 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586666 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586669 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586672 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586675 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586678 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586681 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586684 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586686 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586689 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586692 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586694 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586697 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 12:46:30.588983 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586700 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586702 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586705 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586708 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586712 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586715 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586718 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586722 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586724 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586727 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586731 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586734 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586737 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586740 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586743 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586745 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586748 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586752 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586755 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586757 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 12:46:30.589467 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586760 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586763 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586765 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586768 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586771 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586773 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586776 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586779 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586781 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586784 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586786 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586789 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586792 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586795 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586797 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586800 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586802 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586805 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586808 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586810 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 12:46:30.589962 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586813 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586816 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586819 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586821 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586824 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586827 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586830 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586833 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586836 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586839 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586841 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586844 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586847 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586850 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586852 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 12:46:30.590490 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.586857 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586968 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586972 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586975 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586978 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586981 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586984 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586987 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586989 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586992 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586995 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.586998 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587000 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587003 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587005 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587008 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587011 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587013 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587016 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587019 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 12:46:30.590868 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587021 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587024 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587026 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587029 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587032 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587034 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587037 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587040 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587044 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587047 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587050 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587052 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587055 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587057 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587060 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587062 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587065 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587068 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587070 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587073 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 12:46:30.591406 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587075 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587078 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587081 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587083 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587086 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587089 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587092 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587094 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587097 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587099 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587102 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587104 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587107 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587110 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587112 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587115 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587117 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587120 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587123 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587126 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 12:46:30.591888 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587129 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587131 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587134 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587136 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587139 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587142 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587144 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587147 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587149 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587152 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587154 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587157 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587160 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587163 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587165 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587168 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587170 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587173 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587175 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587178 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 12:46:30.592386 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587182 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587185 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587188 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587191 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587194 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587197 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587200 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587204 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587207 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587210 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587213 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587216 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587219 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587222 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587225 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587228 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587230 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587233 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587235 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 12:46:30.592931 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:30.587238 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 12:46:30.593435 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.587243 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 06 12:46:30.593435 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.587349 2580 server.go:962] "Client rotation is on, will bootstrap in background" Sep 06 12:46:30.593435 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.589621 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 06 12:46:30.593435 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.590599 2580 server.go:1019] "Starting client certificate rotation" Sep 06 12:46:30.593435 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.590698 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 06 12:46:30.593435 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.590728 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 06 12:46:30.593435 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.593229 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 06 12:46:30.597536 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.597520 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 06 12:46:30.608924 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.608900 2580 log.go:25] "Validated CRI v1 runtime API" Sep 06 12:46:30.612109 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.612094 2580 log.go:25] "Validated CRI v1 image API" Sep 06 12:46:30.612951 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.612936 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 06 12:46:30.622632 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.622612 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 06 12:46:30.622920 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.622900 2580 fs.go:135] Filesystem UUIDs: map[21302f7f-8f04-4f90-a9d9-ef7cac8c0d3e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f4b86d2e-4abd-49c3-a293-6ad2eed357d6:/dev/nvme0n1p3] Sep 06 12:46:30.622986 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.622921 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 06 12:46:30.628745 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.628631 2580 manager.go:216] Machine: {Timestamp:2026-09-06 12:46:30.62735927 +0000 UTC m=+0.369730415 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098768 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ea24cc13b3843a73e370f777ddf84 SystemUUID:ec2ea24c-c13b-3843-a73e-370f777ddf84 BootID:8c643234-1ea0-410e-9e4d-0ac0827fca26 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9c:a4:14:13:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9c:a4:14:13:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:22:1c:4b:dd:89 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 06 12:46:30.628745 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.628733 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 06 12:46:30.628913 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.628840 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 06 12:46:30.631945 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.631923 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 06 12:46:30.632075 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.631947 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-13.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 06 12:46:30.633172 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.633162 2580 topology_manager.go:138] "Creating topology manager with none policy" Sep 06 12:46:30.633211 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.633174 2580 container_manager_linux.go:309] "Creating device plugin manager" Sep 06 12:46:30.633211 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.633183 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 06 12:46:30.633822 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.633812 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 06 12:46:30.635074 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.635065 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 06 12:46:30.638582 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.638570 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 06 12:46:30.638690 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.638681 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 06 12:46:30.638729 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.638709 2580 kubelet.go:486] "Attempting to sync node with API server" Sep 06 12:46:30.638729 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.638723 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 06 12:46:30.638791 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.638736 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 06 12:46:30.638791 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.638746 2580 kubelet.go:398] "Adding apiserver pod source" Sep 06 12:46:30.638791 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.638754 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 06 12:46:30.639124 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.639101 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 06 12:46:30.639124 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.639125 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 06 12:46:30.642752 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.642731 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 06 12:46:30.645701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.645687 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 06 12:46:30.645778 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.645716 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 06 12:46:30.645866 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.645857 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 06 12:46:30.645900 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.645872 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 06 12:46:30.645900 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.645878 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 06 12:46:30.645900 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.645884 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 06 12:46:30.645900 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.645892 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 06 12:46:30.645900 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.645897 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 06 12:46:30.646687 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.646677 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 06 12:46:30.646687 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.646687 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 06 12:46:30.646750 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.646694 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 06 12:46:30.646750 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.646700 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 06 12:46:30.646750 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.646708 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 06 12:46:30.646750 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.646717 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 06 12:46:30.647429 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.647416 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 06 12:46:30.647467 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.647432 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 06 12:46:30.651227 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.651210 2580 server.go:1268] "Started kubelet" Sep 06 12:46:30.651480 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.651392 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 06 12:46:30.651991 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.651926 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 06 12:46:30.652080 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.652015 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 06 12:46:30.652172 ip-10-0-141-13 systemd[1]: Started Kubernetes Kubelet. Sep 06 12:46:30.653907 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.653885 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 06 12:46:30.655800 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.655750 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-13.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 06 12:46:30.656093 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.656054 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-13.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 06 12:46:30.656214 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.656138 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 06 12:46:30.658091 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.658067 2580 server.go:310] "Adding debug handlers to kubelet server" Sep 06 12:46:30.658180 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.658071 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 06 12:46:30.661818 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.661800 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 06 12:46:30.662433 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.662406 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 06 12:46:30.662606 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.662589 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:30.663021 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.661694 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-13.ec2.internal.18d2bc12baef2cfa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-13.ec2.internal,UID:ip-10-0-141-13.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-13.ec2.internal,},FirstTimestamp:2026-09-06 12:46:30.651178234 +0000 UTC m=+0.393549428,LastTimestamp:2026-09-06 12:46:30.651178234 +0000 UTC m=+0.393549428,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-13.ec2.internal,}" Sep 06 12:46:30.663241 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.663223 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 06 12:46:30.663241 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.663228 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 06 12:46:30.663395 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.663255 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 06 12:46:30.663395 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.663337 2580 reconstruct.go:97] "Volume reconstruction finished" Sep 06 12:46:30.663395 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.663344 2580 reconciler.go:29] "Reconciler: start to sync state" Sep 06 12:46:30.663395 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.663389 2580 factory.go:55] Registering systemd factory Sep 06 12:46:30.663550 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.663405 2580 factory.go:223] Registration of the systemd container factory successfully Sep 06 12:46:30.663658 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.663642 2580 factory.go:153] Registering CRI-O factory Sep 06 12:46:30.663658 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.663656 2580 factory.go:223] Registration of the crio container factory successfully Sep 06 12:46:30.663757 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.663694 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 06 12:46:30.663757 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.663708 2580 factory.go:103] Registering Raw factory Sep 06 12:46:30.663757 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.663718 2580 manager.go:1196] Started watching for new ooms in manager Sep 06 12:46:30.664139 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.664122 2580 manager.go:349] Starting recovery of all containers Sep 06 12:46:30.664800 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.664776 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-13.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 06 12:46:30.664873 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.664858 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 06 12:46:30.667212 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.667194 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8xxp4" Sep 06 12:46:30.674680 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.674540 2580 manager.go:354] Recovery completed Sep 06 12:46:30.675362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.675343 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8xxp4" Sep 06 12:46:30.681015 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.680999 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 12:46:30.684603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.684587 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientMemory" Sep 06 12:46:30.684678 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.684615 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 12:46:30.684678 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.684625 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientPID" Sep 06 12:46:30.685220 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.685192 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 06 12:46:30.685220 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.685208 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 06 12:46:30.685220 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.685223 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 06 12:46:30.685419 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.685198 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 12:46:30.686037 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.686018 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientMemory" Sep 06 12:46:30.686120 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.686053 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 12:46:30.686120 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.686066 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientPID" Sep 06 12:46:30.686223 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.686212 2580 policy_none.go:49] "None policy: Start" Sep 06 12:46:30.686254 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.686228 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 06 12:46:30.686254 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.686242 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 06 12:46:30.686532 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.686465 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-13.ec2.internal.18d2bc12bced2f13 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-13.ec2.internal,UID:ip-10-0-141-13.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-13.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-13.ec2.internal,},FirstTimestamp:2026-09-06 12:46:30.684602131 +0000 UTC m=+0.426973276,LastTimestamp:2026-09-06 12:46:30.684602131 +0000 UTC m=+0.426973276,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-13.ec2.internal,}" Sep 06 12:46:30.687197 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.687185 2580 policy_none.go:47] "Start" Sep 06 12:46:30.728297 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.728254 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 06 12:46:30.729380 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.729367 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 06 12:46:30.729439 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.729417 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 06 12:46:30.729477 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.729441 2580 kubelet.go:2448] "Starting kubelet main sync loop" Sep 06 12:46:30.729515 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.729482 2580 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Sep 06 12:46:30.731305 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.731286 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 12:46:30.763027 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.762991 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:30.830318 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.830228 2580 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Sep 06 12:46:30.863814 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.863797 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:30.869780 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.869765 2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-13.ec2.internal\" not found" node="ip-10-0-141-13.ec2.internal" Sep 06 12:46:30.873204 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.873192 2580 manager.go:337] "Starting Device Plugin manager" Sep 06 12:46:30.873255 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.873232 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 06 12:46:30.873255 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.873245 2580 server.go:87] "Starting device plugin registration server" Sep 06 12:46:30.873609 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.873595 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 06 12:46:30.873704 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.873607 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 06 12:46:30.873752 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.873712 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 06 12:46:30.873799 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.873793 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 06 12:46:30.873846 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.873804 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 06 12:46:30.874587 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.874568 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 06 12:46:30.874689 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.874610 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:30.974076 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.974038 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 12:46:30.975081 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.975059 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientMemory" Sep 06 12:46:30.975200 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.975096 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 12:46:30.975200 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.975110 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientPID" Sep 06 12:46:30.975200 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.975138 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-13.ec2.internal" Sep 06 12:46:30.984211 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:30.984194 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-13.ec2.internal" Sep 06 12:46:30.984295 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:30.984218 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-13.ec2.internal\": node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:31.008068 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:31.008044 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:31.031126 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.031105 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-13.ec2.internal"] Sep 06 12:46:31.031189 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.031183 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 12:46:31.032787 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.032772 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientMemory" Sep 06 12:46:31.032860 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.032800 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 12:46:31.032860 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.032809 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientPID" Sep 06 12:46:31.034464 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.034451 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 12:46:31.034605 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.034592 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.034646 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.034628 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 12:46:31.035280 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.035250 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientMemory" Sep 06 12:46:31.035280 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.035278 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientMemory" Sep 06 12:46:31.035413 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.035297 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 12:46:31.035413 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.035315 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientPID" Sep 06 12:46:31.035413 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.035299 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 12:46:31.035413 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.035391 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientPID" Sep 06 12:46:31.036950 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.036934 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.037018 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.036961 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 12:46:31.037675 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.037658 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientMemory" Sep 06 12:46:31.037762 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.037686 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 12:46:31.037762 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.037701 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeHasSufficientPID" Sep 06 12:46:31.060969 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:31.060949 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-13.ec2.internal\" not found" node="ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.065276 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.065243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d78411b1fc49eedb8549f17b942dfb68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal\" (UID: \"d78411b1fc49eedb8549f17b942dfb68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.065350 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.065285 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d78411b1fc49eedb8549f17b942dfb68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal\" (UID: \"d78411b1fc49eedb8549f17b942dfb68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.065350 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.065335 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/656e14dae41de34e23352cc32ee02da6-config\") pod \"kube-apiserver-proxy-ip-10-0-141-13.ec2.internal\" (UID: \"656e14dae41de34e23352cc32ee02da6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.065440 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:31.065407 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-13.ec2.internal\" not found" node="ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.108856 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:31.108788 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:31.165448 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.165420 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d78411b1fc49eedb8549f17b942dfb68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal\" (UID: \"d78411b1fc49eedb8549f17b942dfb68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.165571 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.165455 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d78411b1fc49eedb8549f17b942dfb68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal\" (UID: \"d78411b1fc49eedb8549f17b942dfb68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.165571 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.165479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/656e14dae41de34e23352cc32ee02da6-config\") pod \"kube-apiserver-proxy-ip-10-0-141-13.ec2.internal\" (UID: \"656e14dae41de34e23352cc32ee02da6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.165571 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.165532 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/656e14dae41de34e23352cc32ee02da6-config\") pod \"kube-apiserver-proxy-ip-10-0-141-13.ec2.internal\" (UID: \"656e14dae41de34e23352cc32ee02da6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.165571 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.165544 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d78411b1fc49eedb8549f17b942dfb68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal\" (UID: \"d78411b1fc49eedb8549f17b942dfb68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.165710 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.165533 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d78411b1fc49eedb8549f17b942dfb68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal\" (UID: \"d78411b1fc49eedb8549f17b942dfb68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.209587 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:31.209546 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:31.310056 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:31.310020 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:31.365415 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.365367 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.369359 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.369341 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.410345 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:31.410308 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:31.510807 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:31.510754 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:31.590257 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.590230 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 06 12:46:31.590740 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.590380 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 06 12:46:31.611460 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:31.611441 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:31.677489 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.677456 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-05 12:41:31 +0000 UTC" deadline="2028-04-08 15:16:22.103862652 +0000 UTC" Sep 06 12:46:31.677489 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.677482 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13922h29m50.426384388s" Sep 06 12:46:31.711741 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:31.711715 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-13.ec2.internal\" not found" Sep 06 12:46:31.808440 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.808410 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 12:46:31.863145 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.863114 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.877409 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.877387 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 06 12:46:31.879461 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.879444 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-13.ec2.internal" Sep 06 12:46:31.888063 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.887999 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 06 12:46:31.951394 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:31.951359 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod656e14dae41de34e23352cc32ee02da6.slice/crio-5c5569e4559c732f4a321325d6183d592d83f21e6906d358391efe33c5a92fbf WatchSource:0}: Error finding container 5c5569e4559c732f4a321325d6183d592d83f21e6906d358391efe33c5a92fbf: Status 404 returned error can't find the container with id 5c5569e4559c732f4a321325d6183d592d83f21e6906d358391efe33c5a92fbf Sep 06 12:46:31.952558 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:31.952530 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78411b1fc49eedb8549f17b942dfb68.slice/crio-46b14918e4b0a93176ca473673c8d46cb53069066fa13aa277c232b25b5331a0 WatchSource:0}: Error finding container 46b14918e4b0a93176ca473673c8d46cb53069066fa13aa277c232b25b5331a0: Status 404 returned error can't find the container with id 46b14918e4b0a93176ca473673c8d46cb53069066fa13aa277c232b25b5331a0 Sep 06 12:46:31.959694 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.959676 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 12:46:31.981685 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:31.981655 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 12:46:32.247862 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.247772 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 12:46:32.459400 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.459362 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 12:46:32.640206 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.640031 2580 apiserver.go:52] "Watching apiserver" Sep 06 12:46:32.649487 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.649463 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 06 12:46:32.650963 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.650925 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-d9f5r","kube-system/kube-apiserver-proxy-ip-10-0-141-13.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal","openshift-multus/multus-additional-cni-plugins-gmxfr","openshift-multus/network-metrics-daemon-5t886","openshift-network-operator/iptables-alerter-lqrnp","kube-system/konnectivity-agent-dhmsm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm","openshift-cluster-node-tuning-operator/tuned-l84gb","openshift-dns/node-resolver-89gcv","openshift-image-registry/node-ca-qrd8m","openshift-multus/multus-rprqz","openshift-network-diagnostics/network-check-target-x89d8","openshift-ovn-kubernetes/ovnkube-node-4p8dj"] Sep 06 12:46:32.652744 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.652718 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dhmsm" Sep 06 12:46:32.654883 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.654859 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:32.654990 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:32.654936 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:32.655316 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.655285 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 06 12:46:32.655413 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.655380 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2t7wq\"" Sep 06 12:46:32.655480 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.655423 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 06 12:46:32.656269 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.656250 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.659332 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.659111 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 06 12:46:32.659332 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.659231 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4zx47\"" Sep 06 12:46:32.659332 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.659315 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 06 12:46:32.659561 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.659539 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 06 12:46:32.659692 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.659670 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 06 12:46:32.659764 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.659734 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 06 12:46:32.659909 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.659892 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 06 12:46:32.661457 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.660731 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.662751 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.662368 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:32.662751 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:32.662448 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:32.662751 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.662686 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 06 12:46:32.663198 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.663170 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 06 12:46:32.663297 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.663195 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 06 12:46:32.663297 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.663234 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zq8kt\"" Sep 06 12:46:32.663396 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.663371 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 06 12:46:32.663455 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.663404 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 06 12:46:32.663512 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.663470 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 06 12:46:32.663712 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.663695 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lqrnp" Sep 06 12:46:32.665026 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.665006 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.666041 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.666017 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 06 12:46:32.666542 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.666521 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 06 12:46:32.667505 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.666807 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n55c5\"" Sep 06 12:46:32.667505 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.666938 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 06 12:46:32.667505 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.667365 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:32.667505 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.667388 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 06 12:46:32.667505 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.667425 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x6t5j\"" Sep 06 12:46:32.667505 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:32.667455 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:32.668271 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.667956 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.668271 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.668162 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.674123 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.674463 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-46ckt\"" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675094 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-kubelet\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675130 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-cni-netd\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675157 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675190 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk4df\" (UniqueName: \"kubernetes.io/projected/42d02b82-c6e5-4400-b240-dd933ab712a1-kube-api-access-wk4df\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675222 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-kubernetes\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675252 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-sysctl-d\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675279 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/375747af-801e-4186-b4b9-54e30483bd7f-tmp\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-cnibin\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675445 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-run-openvswitch\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675476 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-cni-bin\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675504 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a8776eb0-c74c-4194-82f9-23a98a1fe812-ovnkube-config\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675542 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-var-lib-kubelet\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675566 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675610 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdtw7\" (UniqueName: \"kubernetes.io/projected/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-kube-api-access-fdtw7\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.678003 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675655 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-etc-openvswitch\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.675999 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb9a3147-0623-4e9a-9e9b-2f42510ffb96-iptables-alerter-script\") pod \"iptables-alerter-lqrnp\" (UID: \"fb9a3147-0623-4e9a-9e9b-2f42510ffb96\") " pod="openshift-network-operator/iptables-alerter-lqrnp" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676049 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676080 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-run-systemd\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-var-lib-openvswitch\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676133 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-node-log\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676160 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a8776eb0-c74c-4194-82f9-23a98a1fe812-env-overrides\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676189 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a8776eb0-c74c-4194-82f9-23a98a1fe812-ovn-node-metrics-cert\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676216 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/375747af-801e-4186-b4b9-54e30483bd7f-etc-tuned\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676254 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff818479-7d03-4a64-8e09-2b094c4011a1-agent-certs\") pod \"konnectivity-agent-dhmsm\" (UID: \"ff818479-7d03-4a64-8e09-2b094c4011a1\") " pod="kube-system/konnectivity-agent-dhmsm" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676286 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff818479-7d03-4a64-8e09-2b094c4011a1-konnectivity-ca\") pod \"konnectivity-agent-dhmsm\" (UID: \"ff818479-7d03-4a64-8e09-2b094c4011a1\") " pod="kube-system/konnectivity-agent-dhmsm" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676317 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb9a3147-0623-4e9a-9e9b-2f42510ffb96-host-slash\") pod \"iptables-alerter-lqrnp\" (UID: \"fb9a3147-0623-4e9a-9e9b-2f42510ffb96\") " pod="openshift-network-operator/iptables-alerter-lqrnp" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-systemd-units\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676458 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-run-ovn-kubernetes\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676526 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-modprobe-d\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676561 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-lib-modules\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.679059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-host\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676690 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-system-cni-dir\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676786 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676805 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-slash\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-run-ovn\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676901 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a8776eb0-c74c-4194-82f9-23a98a1fe812-ovnkube-script-lib\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.676975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n97x\" (UniqueName: \"kubernetes.io/projected/a8776eb0-c74c-4194-82f9-23a98a1fe812-kube-api-access-4n97x\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.677075 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm94s\" (UniqueName: \"kubernetes.io/projected/375747af-801e-4186-b4b9-54e30483bd7f-kube-api-access-rm94s\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.677144 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.677188 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s8gd\" (UniqueName: \"kubernetes.io/projected/fb9a3147-0623-4e9a-9e9b-2f42510ffb96-kube-api-access-7s8gd\") pod \"iptables-alerter-lqrnp\" (UID: \"fb9a3147-0623-4e9a-9e9b-2f42510ffb96\") " pod="openshift-network-operator/iptables-alerter-lqrnp" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.677255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm4q2\" (UniqueName: \"kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2\") pod \"network-check-target-x89d8\" (UID: \"0027ef15-df27-4284-9cec-5fa0b603abf1\") " pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.677320 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-run-netns\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.678354 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-89gcv" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.678479 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-sysconfig\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.678530 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-sysctl-conf\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.678572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-run\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.678649 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-sys\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.679858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.678688 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.680756 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.678732 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-log-socket\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.680756 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.678787 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.680756 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.678829 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-systemd\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.680756 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.679286 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 06 12:46:32.680756 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.679682 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qrd8m" Sep 06 12:46:32.680756 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.679824 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 06 12:46:32.681815 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.681218 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 06 12:46:32.681815 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.681605 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rtdkl\"" Sep 06 12:46:32.681815 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.681620 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 06 12:46:32.682772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.682155 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 06 12:46:32.682772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.682507 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 06 12:46:32.682772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.682574 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9kq4d\"" Sep 06 12:46:32.682772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.682651 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 06 12:46:32.683700 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.683683 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.691675 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.687072 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dphth\"" Sep 06 12:46:32.691675 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.687905 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 06 12:46:32.698401 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.698380 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zmf22" Sep 06 12:46:32.706646 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.706629 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zmf22" Sep 06 12:46:32.739985 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.739914 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" event={"ID":"d78411b1fc49eedb8549f17b942dfb68","Type":"ContainerStarted","Data":"46b14918e4b0a93176ca473673c8d46cb53069066fa13aa277c232b25b5331a0"} Sep 06 12:46:32.742122 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.742092 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-13.ec2.internal" event={"ID":"656e14dae41de34e23352cc32ee02da6","Type":"ContainerStarted","Data":"5c5569e4559c732f4a321325d6183d592d83f21e6906d358391efe33c5a92fbf"} Sep 06 12:46:32.764799 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.764776 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 06 12:46:32.779515 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779469 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-cni-bin\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.779651 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779527 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a8776eb0-c74c-4194-82f9-23a98a1fe812-ovnkube-config\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.779651 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779563 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-dbus\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:32.779651 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779613 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-socket-dir\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.779651 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779638 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-device-dir\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.779651 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779646 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-cni-bin\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.779922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779675 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm6k8\" (UniqueName: \"kubernetes.io/projected/987932ec-426e-496d-9da1-506be3f16b92-kube-api-access-jm6k8\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.779922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-etc-openvswitch\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.779922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779733 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3568c869-d959-469d-be7e-c73faa22d4d9-hosts-file\") pod \"node-resolver-89gcv\" (UID: \"3568c869-d959-469d-be7e-c73faa22d4d9\") " pod="openshift-dns/node-resolver-89gcv" Sep 06 12:46:32.779922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779756 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:32.779922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779781 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-etc-selinux\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.779922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779804 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66dvk\" (UniqueName: \"kubernetes.io/projected/5c30ce1d-31d0-4c59-b917-59e79bc7093f-kube-api-access-66dvk\") pod \"node-ca-qrd8m\" (UID: \"5c30ce1d-31d0-4c59-b917-59e79bc7093f\") " pod="openshift-image-registry/node-ca-qrd8m" Sep 06 12:46:32.779922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779825 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-run-k8s-cni-cncf-io\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.779922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779858 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-run-systemd\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.779922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-var-lib-openvswitch\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.779922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779898 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-etc-openvswitch\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.780491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a8776eb0-c74c-4194-82f9-23a98a1fe812-ovn-node-metrics-cert\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.780491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779957 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-var-lib-openvswitch\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.780491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779971 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-run-systemd\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.780491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.779980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff818479-7d03-4a64-8e09-2b094c4011a1-konnectivity-ca\") pod \"konnectivity-agent-dhmsm\" (UID: \"ff818479-7d03-4a64-8e09-2b094c4011a1\") " pod="kube-system/konnectivity-agent-dhmsm" Sep 06 12:46:32.780491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780120 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-multus-cni-dir\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.780491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-kubelet-config\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:32.780491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780320 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb9a3147-0623-4e9a-9e9b-2f42510ffb96-host-slash\") pod \"iptables-alerter-lqrnp\" (UID: \"fb9a3147-0623-4e9a-9e9b-2f42510ffb96\") " pod="openshift-network-operator/iptables-alerter-lqrnp" Sep 06 12:46:32.780491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-systemd-units\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.780491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780392 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-run-ovn-kubernetes\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.780491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780444 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-lib-modules\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.780491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-host\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780498 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-system-cni-dir\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780554 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-systemd-units\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-run-ovn\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780669 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm94s\" (UniqueName: \"kubernetes.io/projected/375747af-801e-4186-b4b9-54e30483bd7f-kube-api-access-rm94s\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780685 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-host\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780758 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-system-cni-dir\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780787 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff818479-7d03-4a64-8e09-2b094c4011a1-konnectivity-ca\") pod \"konnectivity-agent-dhmsm\" (UID: \"ff818479-7d03-4a64-8e09-2b094c4011a1\") " pod="kube-system/konnectivity-agent-dhmsm" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780860 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb9a3147-0623-4e9a-9e9b-2f42510ffb96-host-slash\") pod \"iptables-alerter-lqrnp\" (UID: \"fb9a3147-0623-4e9a-9e9b-2f42510ffb96\") " pod="openshift-network-operator/iptables-alerter-lqrnp" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780896 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-run-ovn\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780958 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-lib-modules\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.780989 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.780986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-run-ovn-kubernetes\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.781462 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781094 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-cnibin\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.781462 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781142 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-var-lib-kubelet\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.781462 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s8gd\" (UniqueName: \"kubernetes.io/projected/fb9a3147-0623-4e9a-9e9b-2f42510ffb96-kube-api-access-7s8gd\") pod \"iptables-alerter-lqrnp\" (UID: \"fb9a3147-0623-4e9a-9e9b-2f42510ffb96\") " pod="openshift-network-operator/iptables-alerter-lqrnp" Sep 06 12:46:32.781462 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781253 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-sysctl-conf\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.781462 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781312 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-sys\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.781462 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781377 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-system-cni-dir\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.781462 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.781723 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781467 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3568c869-d959-469d-be7e-c73faa22d4d9-tmp-dir\") pod \"node-resolver-89gcv\" (UID: \"3568c869-d959-469d-be7e-c73faa22d4d9\") " pod="openshift-dns/node-resolver-89gcv" Sep 06 12:46:32.781723 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781473 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-sysctl-conf\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.781723 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781572 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-sys\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.781723 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781576 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-registration-dir\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.781723 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781614 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.781723 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781620 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/987932ec-426e-496d-9da1-506be3f16b92-multus-daemon-config\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.781723 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781667 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-etc-kubernetes\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.781723 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781696 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-cni-netd\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.781723 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781725 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:32.782053 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781751 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk4df\" (UniqueName: \"kubernetes.io/projected/42d02b82-c6e5-4400-b240-dd933ab712a1-kube-api-access-wk4df\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:32.782053 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-kubernetes\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.782053 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781810 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/375747af-801e-4186-b4b9-54e30483bd7f-tmp\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.782053 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.781947 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-kubernetes\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.782053 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:32.782035 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:46:32.782278 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782056 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-cni-netd\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.782278 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:32.782115 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs podName:42d02b82-c6e5-4400-b240-dd933ab712a1 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:33.282078101 +0000 UTC m=+3.024449242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs") pod "network-metrics-daemon-5t886" (UID: "42d02b82-c6e5-4400-b240-dd933ab712a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:46:32.782278 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782118 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a8776eb0-c74c-4194-82f9-23a98a1fe812-ovnkube-config\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.782278 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782138 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-cnibin\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.782278 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782168 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-run-openvswitch\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.782278 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782194 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-var-lib-kubelet\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.782278 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782220 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.782278 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782254 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-sys-fs\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.782603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782281 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-run-netns\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.782603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782309 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-var-lib-cni-bin\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.782603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782350 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-var-lib-kubelet\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.782603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782359 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-var-lib-cni-multus\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.782603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782395 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-cnibin\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.782603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782399 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb9a3147-0623-4e9a-9e9b-2f42510ffb96-iptables-alerter-script\") pod \"iptables-alerter-lqrnp\" (UID: \"fb9a3147-0623-4e9a-9e9b-2f42510ffb96\") " pod="openshift-network-operator/iptables-alerter-lqrnp" Sep 06 12:46:32.782603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782433 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.782603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782462 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqggh\" (UniqueName: \"kubernetes.io/projected/3568c869-d959-469d-be7e-c73faa22d4d9-kube-api-access-jqggh\") pod \"node-resolver-89gcv\" (UID: \"3568c869-d959-469d-be7e-c73faa22d4d9\") " pod="openshift-dns/node-resolver-89gcv" Sep 06 12:46:32.782603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-hostroot\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.782603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782539 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdtw7\" (UniqueName: \"kubernetes.io/projected/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-kube-api-access-fdtw7\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.782603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-node-log\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.782603 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782590 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a8776eb0-c74c-4194-82f9-23a98a1fe812-env-overrides\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782631 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/375747af-801e-4186-b4b9-54e30483bd7f-etc-tuned\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff818479-7d03-4a64-8e09-2b094c4011a1-agent-certs\") pod \"konnectivity-agent-dhmsm\" (UID: \"ff818479-7d03-4a64-8e09-2b094c4011a1\") " pod="kube-system/konnectivity-agent-dhmsm" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782691 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-run-multus-certs\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-modprobe-d\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782742 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/987932ec-426e-496d-9da1-506be3f16b92-cni-binary-copy\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782766 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-slash\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782799 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a8776eb0-c74c-4194-82f9-23a98a1fe812-ovnkube-script-lib\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782824 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n97x\" (UniqueName: \"kubernetes.io/projected/a8776eb0-c74c-4194-82f9-23a98a1fe812-kube-api-access-4n97x\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782862 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782877 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-multus-conf-dir\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782876 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782920 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm4q2\" (UniqueName: \"kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2\") pod \"network-check-target-x89d8\" (UID: \"0027ef15-df27-4284-9cec-5fa0b603abf1\") " pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-run-netns\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782976 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-sysconfig\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782996 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-run\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.783036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783018 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.783732 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783062 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-multus-socket-dir-parent\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.783732 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-log-socket\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.783732 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-systemd\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.783732 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4kvs\" (UniqueName: \"kubernetes.io/projected/113274a2-bd0c-4ef9-927c-b7347433a60b-kube-api-access-p4kvs\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.783732 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783170 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-kubelet\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.783732 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783194 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-sysctl-d\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.783732 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783219 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c30ce1d-31d0-4c59-b917-59e79bc7093f-host\") pod \"node-ca-qrd8m\" (UID: \"5c30ce1d-31d0-4c59-b917-59e79bc7093f\") " pod="openshift-image-registry/node-ca-qrd8m" Sep 06 12:46:32.783732 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c30ce1d-31d0-4c59-b917-59e79bc7093f-serviceca\") pod \"node-ca-qrd8m\" (UID: \"5c30ce1d-31d0-4c59-b917-59e79bc7093f\") " pod="openshift-image-registry/node-ca-qrd8m" Sep 06 12:46:32.783732 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783498 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb9a3147-0623-4e9a-9e9b-2f42510ffb96-iptables-alerter-script\") pod \"iptables-alerter-lqrnp\" (UID: \"fb9a3147-0623-4e9a-9e9b-2f42510ffb96\") " pod="openshift-network-operator/iptables-alerter-lqrnp" Sep 06 12:46:32.783732 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783725 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.784140 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783766 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.784140 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783795 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-log-socket\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.784140 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.783902 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-kubelet\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.784140 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.784017 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-run-netns\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.784140 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.784058 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-sysctl-d\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.784140 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.784075 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-sysconfig\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.784140 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.784133 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-run\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.784465 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.784171 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-systemd\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.784465 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.784220 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-node-log\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.784465 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.784298 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.784465 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.782435 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-run-openvswitch\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.784627 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.784483 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8776eb0-c74c-4194-82f9-23a98a1fe812-host-slash\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.784627 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.784590 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/375747af-801e-4186-b4b9-54e30483bd7f-etc-modprobe-d\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.785449 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.784732 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a8776eb0-c74c-4194-82f9-23a98a1fe812-env-overrides\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.785449 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.785378 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a8776eb0-c74c-4194-82f9-23a98a1fe812-ovnkube-script-lib\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.789088 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.788393 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/375747af-801e-4186-b4b9-54e30483bd7f-etc-tuned\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.789578 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:32.789293 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 12:46:32.789578 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:32.789318 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 12:46:32.789578 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:32.789333 2580 projected.go:196] Error preparing data for projected volume kube-api-access-cm4q2 for pod openshift-network-diagnostics/network-check-target-x89d8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:46:32.789578 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:32.789415 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2 podName:0027ef15-df27-4284-9cec-5fa0b603abf1 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:33.289395318 +0000 UTC m=+3.031766469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cm4q2" (UniqueName: "kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2") pod "network-check-target-x89d8" (UID: "0027ef15-df27-4284-9cec-5fa0b603abf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:46:32.790742 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.790662 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a8776eb0-c74c-4194-82f9-23a98a1fe812-ovn-node-metrics-cert\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.791381 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.791260 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff818479-7d03-4a64-8e09-2b094c4011a1-agent-certs\") pod \"konnectivity-agent-dhmsm\" (UID: \"ff818479-7d03-4a64-8e09-2b094c4011a1\") " pod="kube-system/konnectivity-agent-dhmsm" Sep 06 12:46:32.799095 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.797454 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/375747af-801e-4186-b4b9-54e30483bd7f-tmp\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.799095 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.797775 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s8gd\" (UniqueName: \"kubernetes.io/projected/fb9a3147-0623-4e9a-9e9b-2f42510ffb96-kube-api-access-7s8gd\") pod \"iptables-alerter-lqrnp\" (UID: \"fb9a3147-0623-4e9a-9e9b-2f42510ffb96\") " pod="openshift-network-operator/iptables-alerter-lqrnp" Sep 06 12:46:32.799095 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.798269 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk4df\" (UniqueName: \"kubernetes.io/projected/42d02b82-c6e5-4400-b240-dd933ab712a1-kube-api-access-wk4df\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:32.799095 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.798687 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm94s\" (UniqueName: \"kubernetes.io/projected/375747af-801e-4186-b4b9-54e30483bd7f-kube-api-access-rm94s\") pod \"tuned-l84gb\" (UID: \"375747af-801e-4186-b4b9-54e30483bd7f\") " pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:32.801631 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.801606 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdtw7\" (UniqueName: \"kubernetes.io/projected/55fb0a7b-617b-477c-ae3a-2c6cf1b33480-kube-api-access-fdtw7\") pod \"multus-additional-cni-plugins-gmxfr\" (UID: \"55fb0a7b-617b-477c-ae3a-2c6cf1b33480\") " pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:32.806744 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.806722 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n97x\" (UniqueName: \"kubernetes.io/projected/a8776eb0-c74c-4194-82f9-23a98a1fe812-kube-api-access-4n97x\") pod \"ovnkube-node-4p8dj\" (UID: \"a8776eb0-c74c-4194-82f9-23a98a1fe812\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:32.884394 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-system-cni-dir\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.884394 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884399 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3568c869-d959-469d-be7e-c73faa22d4d9-tmp-dir\") pod \"node-resolver-89gcv\" (UID: \"3568c869-d959-469d-be7e-c73faa22d4d9\") " pod="openshift-dns/node-resolver-89gcv" Sep 06 12:46:32.884648 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-registration-dir\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.884648 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884452 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/987932ec-426e-496d-9da1-506be3f16b92-multus-daemon-config\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.884648 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884481 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-etc-kubernetes\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.884648 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-sys-fs\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.884648 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-run-netns\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.884648 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-var-lib-cni-bin\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.884648 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884607 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-var-lib-cni-multus\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.884648 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884635 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqggh\" (UniqueName: \"kubernetes.io/projected/3568c869-d959-469d-be7e-c73faa22d4d9-kube-api-access-jqggh\") pod \"node-resolver-89gcv\" (UID: \"3568c869-d959-469d-be7e-c73faa22d4d9\") " pod="openshift-dns/node-resolver-89gcv" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884659 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-hostroot\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884688 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-run-multus-certs\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/987932ec-426e-496d-9da1-506be3f16b92-cni-binary-copy\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884745 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-multus-conf-dir\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884786 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-multus-socket-dir-parent\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884813 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4kvs\" (UniqueName: \"kubernetes.io/projected/113274a2-bd0c-4ef9-927c-b7347433a60b-kube-api-access-p4kvs\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884840 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c30ce1d-31d0-4c59-b917-59e79bc7093f-host\") pod \"node-ca-qrd8m\" (UID: \"5c30ce1d-31d0-4c59-b917-59e79bc7093f\") " pod="openshift-image-registry/node-ca-qrd8m" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c30ce1d-31d0-4c59-b917-59e79bc7093f-serviceca\") pod \"node-ca-qrd8m\" (UID: \"5c30ce1d-31d0-4c59-b917-59e79bc7093f\") " pod="openshift-image-registry/node-ca-qrd8m" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-dbus\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-socket-dir\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884947 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-device-dir\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884972 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm6k8\" (UniqueName: \"kubernetes.io/projected/987932ec-426e-496d-9da1-506be3f16b92-kube-api-access-jm6k8\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.884997 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3568c869-d959-469d-be7e-c73faa22d4d9-hosts-file\") pod \"node-resolver-89gcv\" (UID: \"3568c869-d959-469d-be7e-c73faa22d4d9\") " pod="openshift-dns/node-resolver-89gcv" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885022 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:32.885062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885047 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-etc-selinux\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.885844 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66dvk\" (UniqueName: \"kubernetes.io/projected/5c30ce1d-31d0-4c59-b917-59e79bc7093f-kube-api-access-66dvk\") pod \"node-ca-qrd8m\" (UID: \"5c30ce1d-31d0-4c59-b917-59e79bc7093f\") " pod="openshift-image-registry/node-ca-qrd8m" Sep 06 12:46:32.885844 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885098 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-run-k8s-cni-cncf-io\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.885844 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-multus-cni-dir\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.885844 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885159 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-kubelet-config\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:32.885844 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.885844 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885227 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-cnibin\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.885844 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885251 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-var-lib-kubelet\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.885844 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885370 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-var-lib-kubelet\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.885844 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885447 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-system-cni-dir\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.885844 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885760 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3568c869-d959-469d-be7e-c73faa22d4d9-tmp-dir\") pod \"node-resolver-89gcv\" (UID: \"3568c869-d959-469d-be7e-c73faa22d4d9\") " pod="openshift-dns/node-resolver-89gcv" Sep 06 12:46:32.885844 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.885829 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-registration-dir\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.886437 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.886360 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/987932ec-426e-496d-9da1-506be3f16b92-multus-daemon-config\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.886501 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.886443 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-etc-kubernetes\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.886559 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.886507 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-sys-fs\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.886559 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.886553 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-run-netns\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.886661 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.886597 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-var-lib-cni-bin\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.886661 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.886641 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-var-lib-cni-multus\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.886930 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.886909 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-hostroot\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.887024 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.886967 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-run-multus-certs\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.887475 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.887452 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/987932ec-426e-496d-9da1-506be3f16b92-cni-binary-copy\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.887565 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.887527 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-multus-conf-dir\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.887634 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.887587 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-multus-socket-dir-parent\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.887845 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.887826 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c30ce1d-31d0-4c59-b917-59e79bc7093f-host\") pod \"node-ca-qrd8m\" (UID: \"5c30ce1d-31d0-4c59-b917-59e79bc7093f\") " pod="openshift-image-registry/node-ca-qrd8m" Sep 06 12:46:32.888280 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.888256 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c30ce1d-31d0-4c59-b917-59e79bc7093f-serviceca\") pod \"node-ca-qrd8m\" (UID: \"5c30ce1d-31d0-4c59-b917-59e79bc7093f\") " pod="openshift-image-registry/node-ca-qrd8m" Sep 06 12:46:32.888371 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.888353 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-dbus\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:32.888474 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.888456 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-socket-dir\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.888551 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.888520 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-device-dir\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.888803 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.888786 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3568c869-d959-469d-be7e-c73faa22d4d9-hosts-file\") pod \"node-resolver-89gcv\" (UID: \"3568c869-d959-469d-be7e-c73faa22d4d9\") " pod="openshift-dns/node-resolver-89gcv" Sep 06 12:46:32.888888 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:32.888876 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 12:46:32.888968 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:32.888932 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret podName:5823a60b-e18f-4f3c-b8f5-07cf5ce58135 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:33.388915021 +0000 UTC m=+3.131286162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret") pod "global-pull-secret-syncer-d9f5r" (UID: "5823a60b-e18f-4f3c-b8f5-07cf5ce58135") : object "kube-system"/"original-pull-secret" not registered Sep 06 12:46:32.889192 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.889163 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-etc-selinux\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.889553 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.889528 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/113274a2-bd0c-4ef9-927c-b7347433a60b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.889652 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.889617 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-multus-cni-dir\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.889709 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.889674 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-kubelet-config\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:32.889871 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.889851 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-cnibin\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.889970 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.889952 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/987932ec-426e-496d-9da1-506be3f16b92-host-run-k8s-cni-cncf-io\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.896096 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.896030 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqggh\" (UniqueName: \"kubernetes.io/projected/3568c869-d959-469d-be7e-c73faa22d4d9-kube-api-access-jqggh\") pod \"node-resolver-89gcv\" (UID: \"3568c869-d959-469d-be7e-c73faa22d4d9\") " pod="openshift-dns/node-resolver-89gcv" Sep 06 12:46:32.902548 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.902525 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm6k8\" (UniqueName: \"kubernetes.io/projected/987932ec-426e-496d-9da1-506be3f16b92-kube-api-access-jm6k8\") pod \"multus-rprqz\" (UID: \"987932ec-426e-496d-9da1-506be3f16b92\") " pod="openshift-multus/multus-rprqz" Sep 06 12:46:32.903101 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.902996 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66dvk\" (UniqueName: \"kubernetes.io/projected/5c30ce1d-31d0-4c59-b917-59e79bc7093f-kube-api-access-66dvk\") pod \"node-ca-qrd8m\" (UID: \"5c30ce1d-31d0-4c59-b917-59e79bc7093f\") " pod="openshift-image-registry/node-ca-qrd8m" Sep 06 12:46:32.903101 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.903027 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4kvs\" (UniqueName: \"kubernetes.io/projected/113274a2-bd0c-4ef9-927c-b7347433a60b-kube-api-access-p4kvs\") pod \"aws-ebs-csi-driver-node-hfvjm\" (UID: \"113274a2-bd0c-4ef9-927c-b7347433a60b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:32.968201 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.968166 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dhmsm" Sep 06 12:46:32.990672 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:32.990643 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:33.002529 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:33.002492 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gmxfr" Sep 06 12:46:33.011436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:33.011410 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lqrnp" Sep 06 12:46:33.019285 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:33.019254 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l84gb" Sep 06 12:46:33.030161 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:33.030136 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" Sep 06 12:46:33.036056 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:33.036032 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-89gcv" Sep 06 12:46:33.044831 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:33.044807 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qrd8m" Sep 06 12:46:33.051413 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:33.051395 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rprqz" Sep 06 12:46:33.289374 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:33.289296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:33.289520 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:33.289457 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:46:33.289562 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:33.289521 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs podName:42d02b82-c6e5-4400-b240-dd933ab712a1 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:34.289506784 +0000 UTC m=+4.031877923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs") pod "network-metrics-daemon-5t886" (UID: "42d02b82-c6e5-4400-b240-dd933ab712a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:46:33.390137 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:33.390096 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm4q2\" (UniqueName: \"kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2\") pod \"network-check-target-x89d8\" (UID: \"0027ef15-df27-4284-9cec-5fa0b603abf1\") " pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:33.390371 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:33.390162 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:33.390371 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:33.390290 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 12:46:33.390464 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:33.390383 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret podName:5823a60b-e18f-4f3c-b8f5-07cf5ce58135 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:34.390364154 +0000 UTC m=+4.132735298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret") pod "global-pull-secret-syncer-d9f5r" (UID: "5823a60b-e18f-4f3c-b8f5-07cf5ce58135") : object "kube-system"/"original-pull-secret" not registered Sep 06 12:46:33.390464 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:33.390305 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 12:46:33.390464 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:33.390434 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 12:46:33.390464 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:33.390459 2580 projected.go:196] Error preparing data for projected volume kube-api-access-cm4q2 for pod openshift-network-diagnostics/network-check-target-x89d8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:46:33.390627 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:33.390521 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2 podName:0027ef15-df27-4284-9cec-5fa0b603abf1 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:34.390508293 +0000 UTC m=+4.132879428 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cm4q2" (UniqueName: "kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2") pod "network-check-target-x89d8" (UID: "0027ef15-df27-4284-9cec-5fa0b603abf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:46:33.710253 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:33.710170 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 12:41:33 +0000 UTC" deadline="2028-03-22 12:46:32.099205303 +0000 UTC" Sep 06 12:46:33.710253 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:33.710203 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13511h59m58.389005083s" Sep 06 12:46:33.982098 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:33.981932 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod375747af_801e_4186_b4b9_54e30483bd7f.slice/crio-c52f3ba11fbf06a9eb8610898cf7587e357cd1a20dbcbda4853b64852f7d3db6 WatchSource:0}: Error finding container c52f3ba11fbf06a9eb8610898cf7587e357cd1a20dbcbda4853b64852f7d3db6: Status 404 returned error can't find the container with id c52f3ba11fbf06a9eb8610898cf7587e357cd1a20dbcbda4853b64852f7d3db6 Sep 06 12:46:34.008289 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:34.007988 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod987932ec_426e_496d_9da1_506be3f16b92.slice/crio-8205d473d6706132e5535d495c3218b17cc22bf2b85a5ed1e210bdb9f0724650 WatchSource:0}: Error finding container 8205d473d6706132e5535d495c3218b17cc22bf2b85a5ed1e210bdb9f0724650: Status 404 returned error can't find the container with id 8205d473d6706132e5535d495c3218b17cc22bf2b85a5ed1e210bdb9f0724650 Sep 06 12:46:34.008383 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:34.008351 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff818479_7d03_4a64_8e09_2b094c4011a1.slice/crio-da798295f1750f3f5a085cde5f77c42871e0ba1128625079744dd7f5b74a2101 WatchSource:0}: Error finding container da798295f1750f3f5a085cde5f77c42871e0ba1128625079744dd7f5b74a2101: Status 404 returned error can't find the container with id da798295f1750f3f5a085cde5f77c42871e0ba1128625079744dd7f5b74a2101 Sep 06 12:46:34.009253 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:34.009228 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113274a2_bd0c_4ef9_927c_b7347433a60b.slice/crio-afe3c08fde597da538e9aa5e026d91af6b5967e1cf595888ab7914a642b91139 WatchSource:0}: Error finding container afe3c08fde597da538e9aa5e026d91af6b5967e1cf595888ab7914a642b91139: Status 404 returned error can't find the container with id afe3c08fde597da538e9aa5e026d91af6b5967e1cf595888ab7914a642b91139 Sep 06 12:46:34.009927 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:34.009887 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8776eb0_c74c_4194_82f9_23a98a1fe812.slice/crio-7dcb6eac439deb2b4e56761c77a266dee0bd8905c165ca1f20fbdc5cf4e64687 WatchSource:0}: Error finding container 7dcb6eac439deb2b4e56761c77a266dee0bd8905c165ca1f20fbdc5cf4e64687: Status 404 returned error can't find the container with id 7dcb6eac439deb2b4e56761c77a266dee0bd8905c165ca1f20fbdc5cf4e64687 Sep 06 12:46:34.011709 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:34.011679 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3568c869_d959_469d_be7e_c73faa22d4d9.slice/crio-472d68ffd0a6d5ef1d4a17246d3502791d746e97a5c0adc5220640767322a14a WatchSource:0}: Error finding container 472d68ffd0a6d5ef1d4a17246d3502791d746e97a5c0adc5220640767322a14a: Status 404 returned error can't find the container with id 472d68ffd0a6d5ef1d4a17246d3502791d746e97a5c0adc5220640767322a14a Sep 06 12:46:34.012744 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:34.012717 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9a3147_0623_4e9a_9e9b_2f42510ffb96.slice/crio-ebc85fe3ab09ba840f13db442b8d8a42e5765aad86907b523db25b7a96b01d9a WatchSource:0}: Error finding container ebc85fe3ab09ba840f13db442b8d8a42e5765aad86907b523db25b7a96b01d9a: Status 404 returned error can't find the container with id ebc85fe3ab09ba840f13db442b8d8a42e5765aad86907b523db25b7a96b01d9a Sep 06 12:46:34.014272 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:34.014240 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c30ce1d_31d0_4c59_b917_59e79bc7093f.slice/crio-dfea104d455ed7db327546204e580c5f8c92b25199d6039ad0a381f0789a6189 WatchSource:0}: Error finding container dfea104d455ed7db327546204e580c5f8c92b25199d6039ad0a381f0789a6189: Status 404 returned error can't find the container with id dfea104d455ed7db327546204e580c5f8c92b25199d6039ad0a381f0789a6189 Sep 06 12:46:34.017009 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:46:34.016675 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55fb0a7b_617b_477c_ae3a_2c6cf1b33480.slice/crio-dce71418a895fb38b015572913087ba62a1acfb54270634c719c9e02ffba74a0 WatchSource:0}: Error finding container dce71418a895fb38b015572913087ba62a1acfb54270634c719c9e02ffba74a0: Status 404 returned error can't find the container with id dce71418a895fb38b015572913087ba62a1acfb54270634c719c9e02ffba74a0 Sep 06 12:46:34.298113 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.298082 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:34.298253 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:34.298235 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:46:34.298324 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:34.298314 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs podName:42d02b82-c6e5-4400-b240-dd933ab712a1 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:36.298297622 +0000 UTC m=+6.040668761 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs") pod "network-metrics-daemon-5t886" (UID: "42d02b82-c6e5-4400-b240-dd933ab712a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:46:34.398980 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.398944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:34.399145 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.399030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm4q2\" (UniqueName: \"kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2\") pod \"network-check-target-x89d8\" (UID: \"0027ef15-df27-4284-9cec-5fa0b603abf1\") " pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:34.399211 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:34.399144 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 12:46:34.399211 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:34.399161 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 12:46:34.399211 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:34.399171 2580 projected.go:196] Error preparing data for projected volume kube-api-access-cm4q2 for pod openshift-network-diagnostics/network-check-target-x89d8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:46:34.399211 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:34.399178 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 12:46:34.399387 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:34.399225 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2 podName:0027ef15-df27-4284-9cec-5fa0b603abf1 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:36.399211041 +0000 UTC m=+6.141582176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cm4q2" (UniqueName: "kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2") pod "network-check-target-x89d8" (UID: "0027ef15-df27-4284-9cec-5fa0b603abf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:46:34.399387 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:34.399297 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret podName:5823a60b-e18f-4f3c-b8f5-07cf5ce58135 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:36.399264404 +0000 UTC m=+6.141635543 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret") pod "global-pull-secret-syncer-d9f5r" (UID: "5823a60b-e18f-4f3c-b8f5-07cf5ce58135") : object "kube-system"/"original-pull-secret" not registered Sep 06 12:46:34.710545 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.710429 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 12:41:33 +0000 UTC" deadline="2028-02-07 12:35:11.428926036 +0000 UTC" Sep 06 12:46:34.710545 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.710500 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12455h48m36.718431421s" Sep 06 12:46:34.731244 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.730463 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:34.731244 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:34.730603 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:34.731244 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.731057 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:34.731244 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:34.731182 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:34.731824 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.731435 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:34.731824 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:34.731539 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:34.752759 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.752721 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l84gb" event={"ID":"375747af-801e-4186-b4b9-54e30483bd7f","Type":"ContainerStarted","Data":"c52f3ba11fbf06a9eb8610898cf7587e357cd1a20dbcbda4853b64852f7d3db6"} Sep 06 12:46:34.757140 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.757106 2580 generic.go:356] "Generic (PLEG): container finished" podID="d78411b1fc49eedb8549f17b942dfb68" containerID="c04e4808bd2a3f1a36efc350076b794f5912bd18b84fc48ec81cc118886d3334" exitCode=0 Sep 06 12:46:34.757293 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.757196 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" event={"ID":"d78411b1fc49eedb8549f17b942dfb68","Type":"ContainerDied","Data":"c04e4808bd2a3f1a36efc350076b794f5912bd18b84fc48ec81cc118886d3334"} Sep 06 12:46:34.765221 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.764722 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-13.ec2.internal" event={"ID":"656e14dae41de34e23352cc32ee02da6","Type":"ContainerStarted","Data":"c8da88d4bfe89b00ab86082eeaac4b26e72cdf4b6ddd0021df4aa0b4b1d29d49"} Sep 06 12:46:34.769169 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.769096 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-89gcv" event={"ID":"3568c869-d959-469d-be7e-c73faa22d4d9","Type":"ContainerStarted","Data":"472d68ffd0a6d5ef1d4a17246d3502791d746e97a5c0adc5220640767322a14a"} Sep 06 12:46:34.775574 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.775512 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dhmsm" event={"ID":"ff818479-7d03-4a64-8e09-2b094c4011a1","Type":"ContainerStarted","Data":"da798295f1750f3f5a085cde5f77c42871e0ba1128625079744dd7f5b74a2101"} Sep 06 12:46:34.799397 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.799311 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rprqz" event={"ID":"987932ec-426e-496d-9da1-506be3f16b92","Type":"ContainerStarted","Data":"8205d473d6706132e5535d495c3218b17cc22bf2b85a5ed1e210bdb9f0724650"} Sep 06 12:46:34.813573 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.813472 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmxfr" event={"ID":"55fb0a7b-617b-477c-ae3a-2c6cf1b33480","Type":"ContainerStarted","Data":"dce71418a895fb38b015572913087ba62a1acfb54270634c719c9e02ffba74a0"} Sep 06 12:46:34.818671 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.818600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qrd8m" event={"ID":"5c30ce1d-31d0-4c59-b917-59e79bc7093f","Type":"ContainerStarted","Data":"dfea104d455ed7db327546204e580c5f8c92b25199d6039ad0a381f0789a6189"} Sep 06 12:46:34.828010 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.827940 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lqrnp" event={"ID":"fb9a3147-0623-4e9a-9e9b-2f42510ffb96","Type":"ContainerStarted","Data":"ebc85fe3ab09ba840f13db442b8d8a42e5765aad86907b523db25b7a96b01d9a"} Sep 06 12:46:34.833773 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.833747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" event={"ID":"a8776eb0-c74c-4194-82f9-23a98a1fe812","Type":"ContainerStarted","Data":"7dcb6eac439deb2b4e56761c77a266dee0bd8905c165ca1f20fbdc5cf4e64687"} Sep 06 12:46:34.846637 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:34.846565 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" event={"ID":"113274a2-bd0c-4ef9-927c-b7347433a60b","Type":"ContainerStarted","Data":"afe3c08fde597da538e9aa5e026d91af6b5967e1cf595888ab7914a642b91139"} Sep 06 12:46:35.877036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:35.876997 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" event={"ID":"d78411b1fc49eedb8549f17b942dfb68","Type":"ContainerStarted","Data":"4a6fbfaf101e40653d1c74f7bbb45225b23ce7348a9a1fe61b103650c35278e4"} Sep 06 12:46:35.892976 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:35.892487 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-13.ec2.internal" podStartSLOduration=3.8924658450000003 podStartE2EDuration="3.892465845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 12:46:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:46:34.787709351 +0000 UTC m=+4.530080511" watchObservedRunningTime="2026-09-06 12:46:35.892465845 +0000 UTC m=+5.634837008" Sep 06 12:46:36.316185 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:36.316073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:36.316388 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:36.316260 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:46:36.316388 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:36.316323 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs podName:42d02b82-c6e5-4400-b240-dd933ab712a1 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:40.316304488 +0000 UTC m=+10.058675626 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs") pod "network-metrics-daemon-5t886" (UID: "42d02b82-c6e5-4400-b240-dd933ab712a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:46:36.417108 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:36.417072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:36.417277 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:36.417169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm4q2\" (UniqueName: \"kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2\") pod \"network-check-target-x89d8\" (UID: \"0027ef15-df27-4284-9cec-5fa0b603abf1\") " pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:36.417384 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:36.417257 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 12:46:36.417384 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:36.417349 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 12:46:36.417384 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:36.417369 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 12:46:36.417384 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:36.417383 2580 projected.go:196] Error preparing data for projected volume kube-api-access-cm4q2 for pod openshift-network-diagnostics/network-check-target-x89d8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:46:36.417593 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:36.417353 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret podName:5823a60b-e18f-4f3c-b8f5-07cf5ce58135 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:40.41732652 +0000 UTC m=+10.159697670 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret") pod "global-pull-secret-syncer-d9f5r" (UID: "5823a60b-e18f-4f3c-b8f5-07cf5ce58135") : object "kube-system"/"original-pull-secret" not registered Sep 06 12:46:36.417593 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:36.417446 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2 podName:0027ef15-df27-4284-9cec-5fa0b603abf1 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:40.417429335 +0000 UTC m=+10.159800477 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cm4q2" (UniqueName: "kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2") pod "network-check-target-x89d8" (UID: "0027ef15-df27-4284-9cec-5fa0b603abf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:46:36.731849 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:36.731234 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:36.731849 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:36.731287 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:36.731849 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:36.731234 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:36.731849 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:36.731381 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:36.731849 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:36.731475 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:36.731849 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:36.731580 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:38.731405 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:38.731372 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:38.731995 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:38.731503 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:38.731995 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:38.731915 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:38.732114 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:38.732019 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:38.732183 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:38.732161 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:38.732298 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:38.732246 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:40.352638 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:40.352589 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:40.353143 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:40.352721 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:46:40.353143 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:40.352771 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs podName:42d02b82-c6e5-4400-b240-dd933ab712a1 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:48.352758124 +0000 UTC m=+18.095129257 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs") pod "network-metrics-daemon-5t886" (UID: "42d02b82-c6e5-4400-b240-dd933ab712a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:46:40.453394 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:40.453338 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm4q2\" (UniqueName: \"kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2\") pod \"network-check-target-x89d8\" (UID: \"0027ef15-df27-4284-9cec-5fa0b603abf1\") " pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:40.453554 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:40.453448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:40.453554 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:40.453518 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 12:46:40.453554 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:40.453542 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 12:46:40.453554 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:40.453555 2580 projected.go:196] Error preparing data for projected volume kube-api-access-cm4q2 for pod openshift-network-diagnostics/network-check-target-x89d8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:46:40.453765 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:40.453623 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2 podName:0027ef15-df27-4284-9cec-5fa0b603abf1 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:48.453601706 +0000 UTC m=+18.195972870 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cm4q2" (UniqueName: "kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2") pod "network-check-target-x89d8" (UID: "0027ef15-df27-4284-9cec-5fa0b603abf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:46:40.453765 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:40.453637 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 12:46:40.453765 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:40.453689 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret podName:5823a60b-e18f-4f3c-b8f5-07cf5ce58135 nodeName:}" failed. No retries permitted until 2026-09-06 12:46:48.453673969 +0000 UTC m=+18.196045107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret") pod "global-pull-secret-syncer-d9f5r" (UID: "5823a60b-e18f-4f3c-b8f5-07cf5ce58135") : object "kube-system"/"original-pull-secret" not registered Sep 06 12:46:40.731350 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:40.731273 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:40.731350 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:40.731322 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:40.731539 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:40.731390 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:40.731539 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:40.731451 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:40.731539 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:40.731501 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:40.731646 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:40.731617 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:42.730702 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:42.730186 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:42.730702 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:42.730183 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:42.730702 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:42.730341 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:42.730702 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:42.730384 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:42.730702 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:42.730504 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:42.730702 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:42.730595 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:44.730305 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:44.730248 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:44.730723 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:44.730367 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:44.730723 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:44.730252 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:44.730723 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:44.730406 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:44.730723 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:44.730468 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:44.730723 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:44.730518 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:46.730516 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:46.730478 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:46.730990 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:46.730478 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:46.730990 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:46.730614 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:46.730990 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:46.730606 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:46.730990 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:46.730766 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:46.730990 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:46.730854 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:48.415371 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:48.415124 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:48.415824 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:48.415299 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:46:48.415824 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:48.415489 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs podName:42d02b82-c6e5-4400-b240-dd933ab712a1 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:04.415469143 +0000 UTC m=+34.157840276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs") pod "network-metrics-daemon-5t886" (UID: "42d02b82-c6e5-4400-b240-dd933ab712a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:46:48.516165 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:48.516131 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm4q2\" (UniqueName: \"kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2\") pod \"network-check-target-x89d8\" (UID: \"0027ef15-df27-4284-9cec-5fa0b603abf1\") " pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:48.516368 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:48.516175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:48.516368 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:48.516328 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 12:46:48.516368 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:48.516356 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 12:46:48.516368 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:48.516367 2580 projected.go:196] Error preparing data for projected volume kube-api-access-cm4q2 for pod openshift-network-diagnostics/network-check-target-x89d8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:46:48.516528 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:48.516374 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 12:46:48.516528 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:48.516428 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2 podName:0027ef15-df27-4284-9cec-5fa0b603abf1 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:04.516412916 +0000 UTC m=+34.258784049 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cm4q2" (UniqueName: "kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2") pod "network-check-target-x89d8" (UID: "0027ef15-df27-4284-9cec-5fa0b603abf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:46:48.516528 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:48.516447 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret podName:5823a60b-e18f-4f3c-b8f5-07cf5ce58135 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:04.516440383 +0000 UTC m=+34.258811516 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret") pod "global-pull-secret-syncer-d9f5r" (UID: "5823a60b-e18f-4f3c-b8f5-07cf5ce58135") : object "kube-system"/"original-pull-secret" not registered Sep 06 12:46:48.730389 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:48.730306 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:48.730389 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:48.730333 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:48.730389 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:48.730306 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:48.730646 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:48.730456 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:48.730646 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:48.730546 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:48.730646 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:48.730625 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:50.734420 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:50.734390 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:50.734843 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:50.734484 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:50.734843 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:50.734499 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:50.734843 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:50.734588 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:50.734843 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:50.734645 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:50.734843 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:50.734703 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:52.730574 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.730403 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:52.731319 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.730419 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:52.731319 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:52.730649 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:52.731319 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.730475 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:52.731319 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:52.730738 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:52.731319 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:52.730817 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:52.913279 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.913227 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-89gcv" event={"ID":"3568c869-d959-469d-be7e-c73faa22d4d9","Type":"ContainerStarted","Data":"d37c94d4d8e693f7c05ec7be23cd990e70e62878c72745d4ba30503d75e68406"} Sep 06 12:46:52.914639 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.914613 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dhmsm" event={"ID":"ff818479-7d03-4a64-8e09-2b094c4011a1","Type":"ContainerStarted","Data":"7c176d4ddaaa86969d8aa49ce0740b8cec2e3785525058e576f060a296fbbf47"} Sep 06 12:46:52.916065 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.916038 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rprqz" event={"ID":"987932ec-426e-496d-9da1-506be3f16b92","Type":"ContainerStarted","Data":"2e8fa80c1b17c51bd744fbc98e9120a09d19569214a439ea36b6c5c2c46c92e1"} Sep 06 12:46:52.917551 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.917528 2580 generic.go:356] "Generic (PLEG): container finished" podID="55fb0a7b-617b-477c-ae3a-2c6cf1b33480" containerID="8b07b1cf6ecd137a9ff0094523e9b54214e021c029a83aae15d6b58a0c7c8046" exitCode=0 Sep 06 12:46:52.917642 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.917588 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmxfr" event={"ID":"55fb0a7b-617b-477c-ae3a-2c6cf1b33480","Type":"ContainerDied","Data":"8b07b1cf6ecd137a9ff0094523e9b54214e021c029a83aae15d6b58a0c7c8046"} Sep 06 12:46:52.919186 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.919167 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qrd8m" event={"ID":"5c30ce1d-31d0-4c59-b917-59e79bc7093f","Type":"ContainerStarted","Data":"6a003c72de8c5978560e75386454539d537a71fe2da3eec8b224e45c10cca57c"} Sep 06 12:46:52.922250 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.922232 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" event={"ID":"a8776eb0-c74c-4194-82f9-23a98a1fe812","Type":"ContainerStarted","Data":"78d542f7ae41bf1454bceac43573f4761eea224cffc922c8e15afc0a254d4536"} Sep 06 12:46:52.922344 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.922309 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" event={"ID":"a8776eb0-c74c-4194-82f9-23a98a1fe812","Type":"ContainerStarted","Data":"62ba5881824ab0307883d905839e2f5dee0945985c8057437dcf1d748938aa0c"} Sep 06 12:46:52.922344 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.922341 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" event={"ID":"a8776eb0-c74c-4194-82f9-23a98a1fe812","Type":"ContainerStarted","Data":"ae109d730f09185c43e6102f0cf8cf7286835ba3f8701686f558271f90334531"} Sep 06 12:46:52.922411 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.922375 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" event={"ID":"a8776eb0-c74c-4194-82f9-23a98a1fe812","Type":"ContainerStarted","Data":"c689ac2895cc34f50cfca46a5233d92b6298fc0d37c6356e650f1bcaf92dcb92"} Sep 06 12:46:52.922411 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.922402 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" event={"ID":"a8776eb0-c74c-4194-82f9-23a98a1fe812","Type":"ContainerStarted","Data":"61790c843b7dde259f8ebd0e7941a7fa56650276ef1e5079f5fbe1d648b35abb"} Sep 06 12:46:52.922469 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.922441 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" event={"ID":"a8776eb0-c74c-4194-82f9-23a98a1fe812","Type":"ContainerStarted","Data":"899ccef41ae510ee607958f40cc1399b41d0d08798c90c0c06d977a68e8aef8b"} Sep 06 12:46:52.924010 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.923985 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" event={"ID":"113274a2-bd0c-4ef9-927c-b7347433a60b","Type":"ContainerStarted","Data":"5c542c8d89ef8c205ecc9714b852fd67ad507ed4892b5da317c49c864ca48179"} Sep 06 12:46:52.925287 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.925258 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l84gb" event={"ID":"375747af-801e-4186-b4b9-54e30483bd7f","Type":"ContainerStarted","Data":"a2d161a73653ec48567f8b168bf3747d362b0d8404ab0f7a779da24d706f1acf"} Sep 06 12:46:52.926551 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.926516 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-13.ec2.internal" podStartSLOduration=20.926481576 podStartE2EDuration="20.926481576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 12:46:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:46:35.892903066 +0000 UTC m=+5.635274225" watchObservedRunningTime="2026-09-06 12:46:52.926481576 +0000 UTC m=+22.668852732" Sep 06 12:46:52.926911 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.926890 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-89gcv" podStartSLOduration=3.968336659 podStartE2EDuration="21.9268827s" totalImagesPullingTime="17.958546041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:46:52.926623364 +0000 UTC m=+22.668994501" watchObservedRunningTime="2026-09-06 12:46:52.9268827 +0000 UTC m=+22.669253855" Sep 06 12:46:52.947609 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.947576 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l84gb" podStartSLOduration=3.975856453 podStartE2EDuration="21.947564156s" totalImagesPullingTime="17.971707703s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:46:52.947094466 +0000 UTC m=+22.689465621" watchObservedRunningTime="2026-09-06 12:46:52.947564156 +0000 UTC m=+22.689935320" Sep 06 12:46:52.961805 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:52.961771 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qrd8m" podStartSLOduration=4.009213394 podStartE2EDuration="21.961759724s" totalImagesPullingTime="17.95254633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:46:52.96137155 +0000 UTC m=+22.703742705" watchObservedRunningTime="2026-09-06 12:46:52.961759724 +0000 UTC m=+22.704131262" Sep 06 12:46:53.054430 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:53.054386 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rprqz" podStartSLOduration=4.02607965 podStartE2EDuration="22.054371288s" totalImagesPullingTime="18.028291638s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:46:53.002067657 +0000 UTC m=+22.744438814" watchObservedRunningTime="2026-09-06 12:46:53.054371288 +0000 UTC m=+22.796742442" Sep 06 12:46:53.073557 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:53.073500 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dhmsm" podStartSLOduration=4.11328702 podStartE2EDuration="22.073477692s" totalImagesPullingTime="17.960190672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:46:53.072727719 +0000 UTC m=+22.815098873" watchObservedRunningTime="2026-09-06 12:46:53.073477692 +0000 UTC m=+22.815848847" Sep 06 12:46:53.561525 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:53.561371 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 06 12:46:53.889239 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:53.889098 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-06T12:46:53.561520943Z","UUID":"44af277d-6230-4001-96ca-3d99aa01050e","Handler":null,"Name":"","Endpoint":""} Sep 06 12:46:53.890537 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:53.890519 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 06 12:46:53.890642 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:53.890548 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 06 12:46:53.929449 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:53.929418 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lqrnp" event={"ID":"fb9a3147-0623-4e9a-9e9b-2f42510ffb96","Type":"ContainerStarted","Data":"c2f86bc59097a7a70fde29e27fbb11ed43aa5c0257cedabf6027e3704af5c9d2"} Sep 06 12:46:53.931951 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:53.931925 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" event={"ID":"113274a2-bd0c-4ef9-927c-b7347433a60b","Type":"ContainerStarted","Data":"e6eefb3af717b5d5b8d0e77c5ea895dfd0310e0c78d455dfddec1afd5fa558d5"} Sep 06 12:46:53.942984 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:53.942931 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lqrnp" podStartSLOduration=4.979563282 podStartE2EDuration="22.942914165s" totalImagesPullingTime="17.963350883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:46:53.942587643 +0000 UTC m=+23.684958797" watchObservedRunningTime="2026-09-06 12:46:53.942914165 +0000 UTC m=+23.685285321" Sep 06 12:46:54.730416 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:54.730381 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:54.730594 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:54.730428 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:54.730594 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:54.730380 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:54.730594 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:54.730522 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:54.730738 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:54.730608 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:54.730738 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:54.730695 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:54.936189 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:54.936103 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" event={"ID":"113274a2-bd0c-4ef9-927c-b7347433a60b","Type":"ContainerStarted","Data":"46e85306ff2369febac3f3c5b19b55b786353a3c406efd9c18143db0aa806cd6"} Sep 06 12:46:54.939620 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:54.939597 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" event={"ID":"a8776eb0-c74c-4194-82f9-23a98a1fe812","Type":"ContainerStarted","Data":"cd36512275a5c093f3eb05a1794ef3eeb2d0b229d9a2115ff29c468d46f085fd"} Sep 06 12:46:54.952253 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:54.952212 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfvjm" podStartSLOduration=3.787234904 podStartE2EDuration="23.952199435s" totalImagesPullingTime="20.164964531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:46:54.952046948 +0000 UTC m=+24.694418103" watchObservedRunningTime="2026-09-06 12:46:54.952199435 +0000 UTC m=+24.694570589" Sep 06 12:46:56.395449 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:56.395412 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dhmsm" Sep 06 12:46:56.395994 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:56.395969 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dhmsm" Sep 06 12:46:56.730718 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:56.730638 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:56.730868 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:56.730638 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:56.730868 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:56.730752 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:56.730868 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:56.730833 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:56.730868 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:56.730640 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:56.731048 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:56.730929 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:56.944784 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:56.944748 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dhmsm" Sep 06 12:46:56.945302 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:56.945283 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dhmsm" Sep 06 12:46:58.729863 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:58.729632 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:46:58.730311 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:58.729659 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:46:58.730311 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:58.729923 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:46:58.730311 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:58.729683 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:46:58.730311 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:58.729963 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:46:58.730311 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:46:58.730064 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:46:58.951182 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:58.951151 2580 generic.go:356] "Generic (PLEG): container finished" podID="55fb0a7b-617b-477c-ae3a-2c6cf1b33480" containerID="75b514fa1a895074dc31ce763472ab3ca03467068b8f596a07b48bb972326c61" exitCode=0 Sep 06 12:46:58.951397 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:58.951233 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmxfr" event={"ID":"55fb0a7b-617b-477c-ae3a-2c6cf1b33480","Type":"ContainerDied","Data":"75b514fa1a895074dc31ce763472ab3ca03467068b8f596a07b48bb972326c61"} Sep 06 12:46:58.954973 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:58.954949 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" event={"ID":"a8776eb0-c74c-4194-82f9-23a98a1fe812","Type":"ContainerStarted","Data":"fb0163a101114ff34b51406b6069d0e6c272a046fff3c38f509019103e751e11"} Sep 06 12:46:58.955392 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:58.955219 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:58.955392 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:58.955303 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:58.955392 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:58.955337 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:58.970078 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:58.970055 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:58.970315 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:58.970295 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:46:58.993540 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:46:58.993465 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" podStartSLOduration=9.980233695999999 podStartE2EDuration="27.993451817s" totalImagesPullingTime="18.013218121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:46:58.993016437 +0000 UTC m=+28.735387593" watchObservedRunningTime="2026-09-06 12:46:58.993451817 +0000 UTC m=+28.735822972" Sep 06 12:47:00.179208 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:00.179176 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5t886"] Sep 06 12:47:00.180074 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:00.179344 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:47:00.180074 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:00.179478 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:47:00.182767 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:00.182740 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d9f5r"] Sep 06 12:47:00.182895 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:00.182854 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:47:00.182975 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:00.182953 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:47:00.183490 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:00.183466 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x89d8"] Sep 06 12:47:00.183596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:00.183560 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:47:00.183673 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:00.183650 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:47:00.961284 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:00.961185 2580 generic.go:356] "Generic (PLEG): container finished" podID="55fb0a7b-617b-477c-ae3a-2c6cf1b33480" containerID="3316e90a4556ee22af03bf2b2f5aae233394e739b8707509e392e4b4f5f40e6e" exitCode=0 Sep 06 12:47:00.961442 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:00.961283 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmxfr" event={"ID":"55fb0a7b-617b-477c-ae3a-2c6cf1b33480","Type":"ContainerDied","Data":"3316e90a4556ee22af03bf2b2f5aae233394e739b8707509e392e4b4f5f40e6e"} Sep 06 12:47:01.730387 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:01.730102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:47:01.730821 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:01.730132 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:47:01.730821 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:01.730419 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:47:01.730821 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:01.730130 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:47:01.730821 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:01.730533 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:47:01.730821 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:01.730634 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:47:02.970097 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:02.970063 2580 generic.go:356] "Generic (PLEG): container finished" podID="55fb0a7b-617b-477c-ae3a-2c6cf1b33480" containerID="7e3e77233daca5f880fbd42b2567262bb8c93464f3bde8ac7ea51700174eeb7f" exitCode=0 Sep 06 12:47:02.970536 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:02.970126 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmxfr" event={"ID":"55fb0a7b-617b-477c-ae3a-2c6cf1b33480","Type":"ContainerDied","Data":"7e3e77233daca5f880fbd42b2567262bb8c93464f3bde8ac7ea51700174eeb7f"} Sep 06 12:47:03.729915 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:03.729832 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:47:03.730059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:03.729831 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:47:03.730059 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:03.729960 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9f5r" podUID="5823a60b-e18f-4f3c-b8f5-07cf5ce58135" Sep 06 12:47:03.730059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:03.729836 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:47:03.730059 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:03.730038 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x89d8" podUID="0027ef15-df27-4284-9cec-5fa0b603abf1" Sep 06 12:47:03.730234 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:03.730107 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5t886" podUID="42d02b82-c6e5-4400-b240-dd933ab712a1" Sep 06 12:47:04.438319 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:04.438276 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:47:04.438697 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:04.438420 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:47:04.438697 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:04.438496 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs podName:42d02b82-c6e5-4400-b240-dd933ab712a1 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:36.438479592 +0000 UTC m=+66.180850737 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs") pod "network-metrics-daemon-5t886" (UID: "42d02b82-c6e5-4400-b240-dd933ab712a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 12:47:04.539072 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:04.539037 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm4q2\" (UniqueName: \"kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2\") pod \"network-check-target-x89d8\" (UID: \"0027ef15-df27-4284-9cec-5fa0b603abf1\") " pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:47:04.539244 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:04.539079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:47:04.539323 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:04.539234 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 12:47:04.539323 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:04.539277 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 12:47:04.539323 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:04.539284 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 12:47:04.539447 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:04.539347 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret podName:5823a60b-e18f-4f3c-b8f5-07cf5ce58135 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:36.539333892 +0000 UTC m=+66.281705025 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret") pod "global-pull-secret-syncer-d9f5r" (UID: "5823a60b-e18f-4f3c-b8f5-07cf5ce58135") : object "kube-system"/"original-pull-secret" not registered Sep 06 12:47:04.539447 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:04.539292 2580 projected.go:196] Error preparing data for projected volume kube-api-access-cm4q2 for pod openshift-network-diagnostics/network-check-target-x89d8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:47:04.539447 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:04.539398 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2 podName:0027ef15-df27-4284-9cec-5fa0b603abf1 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:36.539385962 +0000 UTC m=+66.281757107 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cm4q2" (UniqueName: "kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2") pod "network-check-target-x89d8" (UID: "0027ef15-df27-4284-9cec-5fa0b603abf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 12:47:05.613880 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.613847 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-13.ec2.internal" event="NodeReady" Sep 06 12:47:05.614379 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.614008 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 06 12:47:05.648059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.648014 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6777498f4c-ccqwz"] Sep 06 12:47:05.656470 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.656445 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-8zhfs"] Sep 06 12:47:05.656605 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.656565 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.660004 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.659971 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 06 12:47:05.660219 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.660203 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 06 12:47:05.660638 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.660603 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 06 12:47:05.660749 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.660661 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tnxck\"" Sep 06 12:47:05.665213 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.664744 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx"] Sep 06 12:47:05.666532 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.666507 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 06 12:47:05.666775 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.666758 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-8zhfs" Sep 06 12:47:05.668813 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.668776 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 06 12:47:05.668955 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.668862 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-std2z\"" Sep 06 12:47:05.669019 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.668984 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 06 12:47:05.677450 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.675205 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-996bm"] Sep 06 12:47:05.677695 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.675788 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:05.680636 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.680186 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sk4dn\"" Sep 06 12:47:05.680636 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.680235 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 06 12:47:05.680636 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.680280 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 06 12:47:05.680636 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.680490 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 06 12:47:05.686846 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.686821 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2"] Sep 06 12:47:05.686990 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.686949 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.689143 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.689120 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 06 12:47:05.689255 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.689165 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-w8c5p\"" Sep 06 12:47:05.689255 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.689165 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 06 12:47:05.689255 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.689234 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 06 12:47:05.689420 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.689234 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 06 12:47:05.699070 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.699043 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 06 12:47:05.704611 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.704580 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-2vfhs"] Sep 06 12:47:05.704852 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.704837 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:05.708803 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.708784 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nzkhb\"" Sep 06 12:47:05.709005 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.708847 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 06 12:47:05.709005 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.708998 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 06 12:47:05.709168 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.708861 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 06 12:47:05.709168 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.708969 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 06 12:47:05.714555 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.714536 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-659dc76f98-xl7g6"] Sep 06 12:47:05.714803 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.714787 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.717514 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.717489 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-hxx5m\"" Sep 06 12:47:05.717624 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.717541 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 06 12:47:05.717974 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.717953 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 06 12:47:05.718066 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.717999 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 06 12:47:05.718108 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.718075 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 06 12:47:05.722480 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.722459 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 06 12:47:05.726869 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.726852 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.726962 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.726857 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-996bm"] Sep 06 12:47:05.727173 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.727027 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6777498f4c-ccqwz"] Sep 06 12:47:05.727417 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.727401 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx"] Sep 06 12:47:05.727494 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.727442 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-2vfhs"] Sep 06 12:47:05.727494 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.727456 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2"] Sep 06 12:47:05.727494 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.727468 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-659dc76f98-xl7g6"] Sep 06 12:47:05.727494 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.727484 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fgcpf"] Sep 06 12:47:05.728811 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.728791 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 06 12:47:05.729095 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.729071 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 06 12:47:05.729205 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.729078 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 06 12:47:05.729255 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.729130 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-st85w\"" Sep 06 12:47:05.729330 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.729133 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 06 12:47:05.729389 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.729137 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 06 12:47:05.729389 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.729143 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 06 12:47:05.738928 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.738883 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght"] Sep 06 12:47:05.739038 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.739020 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:47:05.739101 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.739045 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:47:05.739275 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.739251 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:47:05.739684 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.739668 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:05.741984 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.741965 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 06 12:47:05.742075 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.742048 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 06 12:47:05.742137 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.742108 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 06 12:47:05.742192 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.742155 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 06 12:47:05.742407 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.742389 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ghsp2\"" Sep 06 12:47:05.742513 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.742469 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 06 12:47:05.742513 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.742495 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g7xpz\"" Sep 06 12:47:05.742629 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.742523 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nqwjz\"" Sep 06 12:47:05.742629 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.742559 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 06 12:47:05.750914 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.750895 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl"] Sep 06 12:47:05.751029 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.750970 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" Sep 06 12:47:05.753174 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.753155 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 06 12:47:05.753596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.753582 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 06 12:47:05.753775 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.753753 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qss95\"" Sep 06 12:47:05.753960 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.753874 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 06 12:47:05.754170 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.754148 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 06 12:47:05.761098 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.761077 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-h8tgn"] Sep 06 12:47:05.761296 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.761226 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" Sep 06 12:47:05.763424 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.763402 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 06 12:47:05.763523 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.763424 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 06 12:47:05.763523 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.763501 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-h7m4h\"" Sep 06 12:47:05.763629 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.763609 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 06 12:47:05.763780 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.763762 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 06 12:47:05.777173 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.777153 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-wccw5"] Sep 06 12:47:05.777321 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.777301 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h8tgn" Sep 06 12:47:05.781871 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.781848 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bhdml\"" Sep 06 12:47:05.782138 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.782120 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 06 12:47:05.782616 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.782365 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 06 12:47:05.782616 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.782516 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 06 12:47:05.793371 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.793350 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-8zhfs"] Sep 06 12:47:05.793469 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.793381 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-wccw5"] Sep 06 12:47:05.793469 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.793398 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght"] Sep 06 12:47:05.793469 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.793417 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h8tgn"] Sep 06 12:47:05.793469 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.793436 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl"] Sep 06 12:47:05.793469 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.793454 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fgcpf"] Sep 06 12:47:05.793707 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.793532 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-wccw5" Sep 06 12:47:05.795678 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.795661 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xlcs4\"" Sep 06 12:47:05.827658 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.827624 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dhmsm_ff818479-7d03-4a64-8e09-2b094c4011a1/konnectivity-agent/0.log" limit=1024 Sep 06 12:47:05.847228 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847199 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c34f36a1-7b51-43f1-a481-e5331b2ee206-config-volume\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:05.847391 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847278 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-certificates\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.847391 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847309 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:05.847391 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847337 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xztbj\" (UniqueName: \"kubernetes.io/projected/8750b827-635a-4805-817c-45a970d9fc81-kube-api-access-xztbj\") pod \"console-operator-85b454874b-2vfhs\" (UID: \"8750b827-635a-4805-817c-45a970d9fc81\") " pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.847531 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847386 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c34f36a1-7b51-43f1-a481-e5331b2ee206-tmp-dir\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:05.847531 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847424 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-default-certificate\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.847531 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847479 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb9nk\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-kube-api-access-mb9nk\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.847710 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847535 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-installation-pull-secrets\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.847710 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847559 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpz5n\" (UniqueName: \"kubernetes.io/projected/c34f36a1-7b51-43f1-a481-e5331b2ee206-kube-api-access-zpz5n\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:05.847710 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847583 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-bound-sa-token\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.847710 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847601 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b93d75ad-6183-460c-9652-c502a31ba231-snapshots\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.847710 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847638 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.847710 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847673 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-trusted-ca\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.847952 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-ca-trust-extracted\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.847952 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93d75ad-6183-460c-9652-c502a31ba231-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.847952 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847880 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqvh8\" (UniqueName: \"kubernetes.io/projected/0c15574a-7aa1-4f24-b884-90ec43811ef2-kube-api-access-sqvh8\") pod \"cluster-samples-operator-7cdcfcf76c-b5zlx\" (UID: \"0c15574a-7aa1-4f24-b884-90ec43811ef2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:05.847952 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847916 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdq5h\" (UniqueName: \"kubernetes.io/projected/75be41fd-be89-4ca5-9bab-ac64e74295ba-kube-api-access-rdq5h\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.847952 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847937 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab6850db-dfc3-47f9-b91b-54757857d601-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-s6ght\" (UID: \"ab6850db-dfc3-47f9-b91b-54757857d601\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" Sep 06 12:47:05.848164 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847958 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8750b827-635a-4805-817c-45a970d9fc81-trusted-ca\") pod \"console-operator-85b454874b-2vfhs\" (UID: \"8750b827-635a-4805-817c-45a970d9fc81\") " pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.848164 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.847982 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b93d75ad-6183-460c-9652-c502a31ba231-tmp\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.848164 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848035 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-image-registry-private-configuration\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.848164 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848059 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93d75ad-6183-460c-9652-c502a31ba231-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.848164 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848079 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6jsr\" (UniqueName: \"kubernetes.io/projected/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-kube-api-access-m6jsr\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:05.848164 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848105 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbqsw\" (UniqueName: \"kubernetes.io/projected/ab6850db-dfc3-47f9-b91b-54757857d601-kube-api-access-sbqsw\") pod \"kube-storage-version-migrator-operator-8d7dd4675-s6ght\" (UID: \"ab6850db-dfc3-47f9-b91b-54757857d601\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" Sep 06 12:47:05.848164 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848151 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qp9k\" (UniqueName: \"kubernetes.io/projected/9c2987eb-10ff-4823-966e-7e65d410fc56-kube-api-access-2qp9k\") pod \"volume-data-source-validator-695c895bd6-8zhfs\" (UID: \"9c2987eb-10ff-4823-966e-7e65d410fc56\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-8zhfs" Sep 06 12:47:05.848596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848189 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8750b827-635a-4805-817c-45a970d9fc81-config\") pod \"console-operator-85b454874b-2vfhs\" (UID: \"8750b827-635a-4805-817c-45a970d9fc81\") " pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.848596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-stats-auth\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.848596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848289 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.848596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848314 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6g9f\" (UniqueName: \"kubernetes.io/projected/b93d75ad-6183-460c-9652-c502a31ba231-kube-api-access-w6g9f\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.848596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848333 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:05.848596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b93d75ad-6183-460c-9652-c502a31ba231-serving-cert\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.848596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848378 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:05.848596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848402 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab6850db-dfc3-47f9-b91b-54757857d601-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-s6ght\" (UID: \"ab6850db-dfc3-47f9-b91b-54757857d601\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" Sep 06 12:47:05.848596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848423 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.848596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848464 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-b5zlx\" (UID: \"0c15574a-7aa1-4f24-b884-90ec43811ef2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:05.848596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.848488 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8750b827-635a-4805-817c-45a970d9fc81-serving-cert\") pod \"console-operator-85b454874b-2vfhs\" (UID: \"8750b827-635a-4805-817c-45a970d9fc81\") " pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.905080 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.905044 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dhmsm_ff818479-7d03-4a64-8e09-2b094c4011a1/konnectivity-agent/0.log" limit=1024 Sep 06 12:47:05.949043 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgwdx\" (UniqueName: \"kubernetes.io/projected/fc4b28cb-5070-41be-81d6-bf4965e44af1-kube-api-access-zgwdx\") pod \"service-ca-operator-859f94c4b-mw7gl\" (UID: \"fc4b28cb-5070-41be-81d6-bf4965e44af1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" Sep 06 12:47:05.949208 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c34f36a1-7b51-43f1-a481-e5331b2ee206-config-volume\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:05.949208 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949102 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f09a5d91-5be0-4cff-8388-d3f33dfb0391-cert\") pod \"ingress-canary-h8tgn\" (UID: \"f09a5d91-5be0-4cff-8388-d3f33dfb0391\") " pod="openshift-ingress-canary/ingress-canary-h8tgn" Sep 06 12:47:05.949208 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-certificates\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.949208 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949158 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:05.949208 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949184 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xztbj\" (UniqueName: \"kubernetes.io/projected/8750b827-635a-4805-817c-45a970d9fc81-kube-api-access-xztbj\") pod \"console-operator-85b454874b-2vfhs\" (UID: \"8750b827-635a-4805-817c-45a970d9fc81\") " pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.949438 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949211 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c34f36a1-7b51-43f1-a481-e5331b2ee206-tmp-dir\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:05.949438 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949238 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-default-certificate\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.949438 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb9nk\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-kube-api-access-mb9nk\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.949438 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949318 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-982zd\" (UniqueName: \"kubernetes.io/projected/f09a5d91-5be0-4cff-8388-d3f33dfb0391-kube-api-access-982zd\") pod \"ingress-canary-h8tgn\" (UID: \"f09a5d91-5be0-4cff-8388-d3f33dfb0391\") " pod="openshift-ingress-canary/ingress-canary-h8tgn" Sep 06 12:47:05.949438 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-installation-pull-secrets\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.949438 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpz5n\" (UniqueName: \"kubernetes.io/projected/c34f36a1-7b51-43f1-a481-e5331b2ee206-kube-api-access-zpz5n\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:05.949438 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-bound-sa-token\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.949438 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b93d75ad-6183-460c-9652-c502a31ba231-snapshots\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.949758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949451 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.949758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949517 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-trusted-ca\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.949758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-ca-trust-extracted\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.949758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949577 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93d75ad-6183-460c-9652-c502a31ba231-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.949758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949606 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqvh8\" (UniqueName: \"kubernetes.io/projected/0c15574a-7aa1-4f24-b884-90ec43811ef2-kube-api-access-sqvh8\") pod \"cluster-samples-operator-7cdcfcf76c-b5zlx\" (UID: \"0c15574a-7aa1-4f24-b884-90ec43811ef2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:05.949758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949635 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdq5h\" (UniqueName: \"kubernetes.io/projected/75be41fd-be89-4ca5-9bab-ac64e74295ba-kube-api-access-rdq5h\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.949758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949661 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc4b28cb-5070-41be-81d6-bf4965e44af1-serving-cert\") pod \"service-ca-operator-859f94c4b-mw7gl\" (UID: \"fc4b28cb-5070-41be-81d6-bf4965e44af1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" Sep 06 12:47:05.949758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949688 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab6850db-dfc3-47f9-b91b-54757857d601-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-s6ght\" (UID: \"ab6850db-dfc3-47f9-b91b-54757857d601\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" Sep 06 12:47:05.949758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8750b827-635a-4805-817c-45a970d9fc81-trusted-ca\") pod \"console-operator-85b454874b-2vfhs\" (UID: \"8750b827-635a-4805-817c-45a970d9fc81\") " pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.949758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949715 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c34f36a1-7b51-43f1-a481-e5331b2ee206-config-volume\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:05.949758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b93d75ad-6183-460c-9652-c502a31ba231-tmp\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.950290 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949780 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-image-registry-private-configuration\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.950290 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949807 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93d75ad-6183-460c-9652-c502a31ba231-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.950290 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949835 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6jsr\" (UniqueName: \"kubernetes.io/projected/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-kube-api-access-m6jsr\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:05.950290 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbqsw\" (UniqueName: \"kubernetes.io/projected/ab6850db-dfc3-47f9-b91b-54757857d601-kube-api-access-sbqsw\") pod \"kube-storage-version-migrator-operator-8d7dd4675-s6ght\" (UID: \"ab6850db-dfc3-47f9-b91b-54757857d601\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" Sep 06 12:47:05.950290 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949878 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:05.950290 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.949896 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qp9k\" (UniqueName: \"kubernetes.io/projected/9c2987eb-10ff-4823-966e-7e65d410fc56-kube-api-access-2qp9k\") pod \"volume-data-source-validator-695c895bd6-8zhfs\" (UID: \"9c2987eb-10ff-4823-966e-7e65d410fc56\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-8zhfs" Sep 06 12:47:05.950531 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950307 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c34f36a1-7b51-43f1-a481-e5331b2ee206-tmp-dir\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:05.950531 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950391 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-certificates\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.950531 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950431 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc4b28cb-5070-41be-81d6-bf4965e44af1-config\") pod \"service-ca-operator-859f94c4b-mw7gl\" (UID: \"fc4b28cb-5070-41be-81d6-bf4965e44af1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" Sep 06 12:47:05.950531 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8750b827-635a-4805-817c-45a970d9fc81-config\") pod \"console-operator-85b454874b-2vfhs\" (UID: \"8750b827-635a-4805-817c-45a970d9fc81\") " pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.950531 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-stats-auth\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.950726 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950538 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.950726 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950563 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n72lv\" (UniqueName: \"kubernetes.io/projected/73ed942b-28b8-492f-ac72-65e2dfa64b25-kube-api-access-n72lv\") pod \"network-check-source-685456655f-wccw5\" (UID: \"73ed942b-28b8-492f-ac72-65e2dfa64b25\") " pod="openshift-network-diagnostics/network-check-source-685456655f-wccw5" Sep 06 12:47:05.950726 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950598 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6g9f\" (UniqueName: \"kubernetes.io/projected/b93d75ad-6183-460c-9652-c502a31ba231-kube-api-access-w6g9f\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.950726 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950619 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:05.950726 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950647 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b93d75ad-6183-460c-9652-c502a31ba231-serving-cert\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.950726 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950670 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:05.950726 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950693 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab6850db-dfc3-47f9-b91b-54757857d601-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-s6ght\" (UID: \"ab6850db-dfc3-47f9-b91b-54757857d601\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" Sep 06 12:47:05.950726 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950713 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.951055 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950738 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-b5zlx\" (UID: \"0c15574a-7aa1-4f24-b884-90ec43811ef2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:05.951055 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.950761 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8750b827-635a-4805-817c-45a970d9fc81-serving-cert\") pod \"console-operator-85b454874b-2vfhs\" (UID: \"8750b827-635a-4805-817c-45a970d9fc81\") " pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.951142 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.951063 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-trusted-ca\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.951903 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.951411 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b93d75ad-6183-460c-9652-c502a31ba231-tmp\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.951903 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.951469 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-ca-trust-extracted\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.951903 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:05.951497 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 06 12:47:05.951903 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:05.951512 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777498f4c-ccqwz: secret "image-registry-tls" not found Sep 06 12:47:05.951903 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:05.951566 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls podName:f220f886-f2be-4f33-b8fc-1d0f3dd3dafd nodeName:}" failed. No retries permitted until 2026-09-06 12:47:06.451552629 +0000 UTC m=+36.193923763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls") pod "image-registry-6777498f4c-ccqwz" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd") : secret "image-registry-tls" not found Sep 06 12:47:05.951903 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.951808 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93d75ad-6183-460c-9652-c502a31ba231-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.951903 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.951809 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8750b827-635a-4805-817c-45a970d9fc81-trusted-ca\") pod \"console-operator-85b454874b-2vfhs\" (UID: \"8750b827-635a-4805-817c-45a970d9fc81\") " pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.952622 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.952316 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8750b827-635a-4805-817c-45a970d9fc81-config\") pod \"console-operator-85b454874b-2vfhs\" (UID: \"8750b827-635a-4805-817c-45a970d9fc81\") " pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.952622 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.952490 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab6850db-dfc3-47f9-b91b-54757857d601-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-s6ght\" (UID: \"ab6850db-dfc3-47f9-b91b-54757857d601\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" Sep 06 12:47:05.952622 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:05.952536 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 06 12:47:05.952622 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:05.952569 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 06 12:47:05.952622 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:05.952599 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls podName:c34f36a1-7b51-43f1-a481-e5331b2ee206 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:06.452582411 +0000 UTC m=+36.194953560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls") pod "dns-default-fgcpf" (UID: "c34f36a1-7b51-43f1-a481-e5331b2ee206") : secret "dns-default-metrics-tls" not found Sep 06 12:47:05.952622 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:05.952623 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs podName:75be41fd-be89-4ca5-9bab-ac64e74295ba nodeName:}" failed. No retries permitted until 2026-09-06 12:47:06.452608348 +0000 UTC m=+36.194979497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs") pod "router-default-659dc76f98-xl7g6" (UID: "75be41fd-be89-4ca5-9bab-ac64e74295ba") : secret "router-metrics-certs-default" not found Sep 06 12:47:05.953060 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.952652 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93d75ad-6183-460c-9652-c502a31ba231-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.953060 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:05.952864 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle podName:75be41fd-be89-4ca5-9bab-ac64e74295ba nodeName:}" failed. No retries permitted until 2026-09-06 12:47:06.452843642 +0000 UTC m=+36.195214797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle") pod "router-default-659dc76f98-xl7g6" (UID: "75be41fd-be89-4ca5-9bab-ac64e74295ba") : configmap references non-existent config key: service-ca.crt Sep 06 12:47:05.953060 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:05.952880 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 06 12:47:05.953060 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.952928 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b93d75ad-6183-460c-9652-c502a31ba231-snapshots\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.953060 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:05.952969 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 06 12:47:05.953060 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:05.952984 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls podName:0c15574a-7aa1-4f24-b884-90ec43811ef2 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:06.452969299 +0000 UTC m=+36.195340445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-b5zlx" (UID: "0c15574a-7aa1-4f24-b884-90ec43811ef2") : secret "samples-operator-tls" not found Sep 06 12:47:05.953060 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:05.953008 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls podName:b8dd16fa-c99c-4c68-b472-4927f82d5e5a nodeName:}" failed. No retries permitted until 2026-09-06 12:47:06.452997465 +0000 UTC m=+36.195368613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9gph2" (UID: "b8dd16fa-c99c-4c68-b472-4927f82d5e5a") : secret "cluster-monitoring-operator-tls" not found Sep 06 12:47:05.955760 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.955700 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-installation-pull-secrets\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.955760 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.955706 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-image-registry-private-configuration\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.955947 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.955926 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8750b827-635a-4805-817c-45a970d9fc81-serving-cert\") pod \"console-operator-85b454874b-2vfhs\" (UID: \"8750b827-635a-4805-817c-45a970d9fc81\") " pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.956017 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.955994 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-default-certificate\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.956332 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.956311 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab6850db-dfc3-47f9-b91b-54757857d601-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-s6ght\" (UID: \"ab6850db-dfc3-47f9-b91b-54757857d601\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" Sep 06 12:47:05.956522 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.956500 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-stats-auth\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.957635 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.957592 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b93d75ad-6183-460c-9652-c502a31ba231-serving-cert\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.959158 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.959135 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb9nk\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-kube-api-access-mb9nk\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.959379 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.959356 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqvh8\" (UniqueName: \"kubernetes.io/projected/0c15574a-7aa1-4f24-b884-90ec43811ef2-kube-api-access-sqvh8\") pod \"cluster-samples-operator-7cdcfcf76c-b5zlx\" (UID: \"0c15574a-7aa1-4f24-b884-90ec43811ef2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:05.960050 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.960028 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qp9k\" (UniqueName: \"kubernetes.io/projected/9c2987eb-10ff-4823-966e-7e65d410fc56-kube-api-access-2qp9k\") pod \"volume-data-source-validator-695c895bd6-8zhfs\" (UID: \"9c2987eb-10ff-4823-966e-7e65d410fc56\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-8zhfs" Sep 06 12:47:05.960953 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.960649 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6g9f\" (UniqueName: \"kubernetes.io/projected/b93d75ad-6183-460c-9652-c502a31ba231-kube-api-access-w6g9f\") pod \"insights-operator-76bcd6ccf5-996bm\" (UID: \"b93d75ad-6183-460c-9652-c502a31ba231\") " pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:05.961063 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.960974 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xztbj\" (UniqueName: \"kubernetes.io/projected/8750b827-635a-4805-817c-45a970d9fc81-kube-api-access-xztbj\") pod \"console-operator-85b454874b-2vfhs\" (UID: \"8750b827-635a-4805-817c-45a970d9fc81\") " pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:05.962469 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.962421 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdq5h\" (UniqueName: \"kubernetes.io/projected/75be41fd-be89-4ca5-9bab-ac64e74295ba-kube-api-access-rdq5h\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:05.962665 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.962643 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6jsr\" (UniqueName: \"kubernetes.io/projected/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-kube-api-access-m6jsr\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:05.963124 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.963103 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbqsw\" (UniqueName: \"kubernetes.io/projected/ab6850db-dfc3-47f9-b91b-54757857d601-kube-api-access-sbqsw\") pod \"kube-storage-version-migrator-operator-8d7dd4675-s6ght\" (UID: \"ab6850db-dfc3-47f9-b91b-54757857d601\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" Sep 06 12:47:05.963320 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.963302 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpz5n\" (UniqueName: \"kubernetes.io/projected/c34f36a1-7b51-43f1-a481-e5331b2ee206-kube-api-access-zpz5n\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:05.963599 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.963577 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-bound-sa-token\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:05.983154 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:05.982977 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-8zhfs" Sep 06 12:47:06.008421 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.008349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" Sep 06 12:47:06.025333 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.025309 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:06.052006 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.051969 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc4b28cb-5070-41be-81d6-bf4965e44af1-serving-cert\") pod \"service-ca-operator-859f94c4b-mw7gl\" (UID: \"fc4b28cb-5070-41be-81d6-bf4965e44af1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" Sep 06 12:47:06.052113 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.052028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc4b28cb-5070-41be-81d6-bf4965e44af1-config\") pod \"service-ca-operator-859f94c4b-mw7gl\" (UID: \"fc4b28cb-5070-41be-81d6-bf4965e44af1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" Sep 06 12:47:06.052113 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.052087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n72lv\" (UniqueName: \"kubernetes.io/projected/73ed942b-28b8-492f-ac72-65e2dfa64b25-kube-api-access-n72lv\") pod \"network-check-source-685456655f-wccw5\" (UID: \"73ed942b-28b8-492f-ac72-65e2dfa64b25\") " pod="openshift-network-diagnostics/network-check-source-685456655f-wccw5" Sep 06 12:47:06.052193 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.052178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgwdx\" (UniqueName: \"kubernetes.io/projected/fc4b28cb-5070-41be-81d6-bf4965e44af1-kube-api-access-zgwdx\") pod \"service-ca-operator-859f94c4b-mw7gl\" (UID: \"fc4b28cb-5070-41be-81d6-bf4965e44af1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" Sep 06 12:47:06.052230 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.052220 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f09a5d91-5be0-4cff-8388-d3f33dfb0391-cert\") pod \"ingress-canary-h8tgn\" (UID: \"f09a5d91-5be0-4cff-8388-d3f33dfb0391\") " pod="openshift-ingress-canary/ingress-canary-h8tgn" Sep 06 12:47:06.052317 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.052291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-982zd\" (UniqueName: \"kubernetes.io/projected/f09a5d91-5be0-4cff-8388-d3f33dfb0391-kube-api-access-982zd\") pod \"ingress-canary-h8tgn\" (UID: \"f09a5d91-5be0-4cff-8388-d3f33dfb0391\") " pod="openshift-ingress-canary/ingress-canary-h8tgn" Sep 06 12:47:06.052686 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.052663 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc4b28cb-5070-41be-81d6-bf4965e44af1-config\") pod \"service-ca-operator-859f94c4b-mw7gl\" (UID: \"fc4b28cb-5070-41be-81d6-bf4965e44af1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" Sep 06 12:47:06.054973 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.054922 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc4b28cb-5070-41be-81d6-bf4965e44af1-serving-cert\") pod \"service-ca-operator-859f94c4b-mw7gl\" (UID: \"fc4b28cb-5070-41be-81d6-bf4965e44af1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" Sep 06 12:47:06.055056 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.054974 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f09a5d91-5be0-4cff-8388-d3f33dfb0391-cert\") pod \"ingress-canary-h8tgn\" (UID: \"f09a5d91-5be0-4cff-8388-d3f33dfb0391\") " pod="openshift-ingress-canary/ingress-canary-h8tgn" Sep 06 12:47:06.059663 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.059621 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgwdx\" (UniqueName: \"kubernetes.io/projected/fc4b28cb-5070-41be-81d6-bf4965e44af1-kube-api-access-zgwdx\") pod \"service-ca-operator-859f94c4b-mw7gl\" (UID: \"fc4b28cb-5070-41be-81d6-bf4965e44af1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" Sep 06 12:47:06.060495 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.060467 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-982zd\" (UniqueName: \"kubernetes.io/projected/f09a5d91-5be0-4cff-8388-d3f33dfb0391-kube-api-access-982zd\") pod \"ingress-canary-h8tgn\" (UID: \"f09a5d91-5be0-4cff-8388-d3f33dfb0391\") " pod="openshift-ingress-canary/ingress-canary-h8tgn" Sep 06 12:47:06.060495 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.060480 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n72lv\" (UniqueName: \"kubernetes.io/projected/73ed942b-28b8-492f-ac72-65e2dfa64b25-kube-api-access-n72lv\") pod \"network-check-source-685456655f-wccw5\" (UID: \"73ed942b-28b8-492f-ac72-65e2dfa64b25\") " pod="openshift-network-diagnostics/network-check-source-685456655f-wccw5" Sep 06 12:47:06.080367 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.080335 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" Sep 06 12:47:06.087529 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.087509 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" Sep 06 12:47:06.093646 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.093624 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h8tgn" Sep 06 12:47:06.103785 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.103761 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-wccw5" Sep 06 12:47:06.455960 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.455924 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:06.456168 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.455977 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:06.456168 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.456011 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:06.456168 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:06.456110 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 06 12:47:06.456168 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.456144 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:06.456168 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:06.456169 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls podName:b8dd16fa-c99c-4c68-b472-4927f82d5e5a nodeName:}" failed. No retries permitted until 2026-09-06 12:47:07.456155571 +0000 UTC m=+37.198526704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9gph2" (UID: "b8dd16fa-c99c-4c68-b472-4927f82d5e5a") : secret "cluster-monitoring-operator-tls" not found Sep 06 12:47:06.456491 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:06.456172 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 06 12:47:06.456491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.456203 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-b5zlx\" (UID: \"0c15574a-7aa1-4f24-b884-90ec43811ef2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:06.456491 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:06.456114 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 06 12:47:06.456491 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:06.456223 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls podName:c34f36a1-7b51-43f1-a481-e5331b2ee206 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:07.45620844 +0000 UTC m=+37.198579590 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls") pod "dns-default-fgcpf" (UID: "c34f36a1-7b51-43f1-a481-e5331b2ee206") : secret "dns-default-metrics-tls" not found Sep 06 12:47:06.456491 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:06.456249 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle podName:75be41fd-be89-4ca5-9bab-ac64e74295ba nodeName:}" failed. No retries permitted until 2026-09-06 12:47:07.456237155 +0000 UTC m=+37.198608289 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle") pod "router-default-659dc76f98-xl7g6" (UID: "75be41fd-be89-4ca5-9bab-ac64e74295ba") : configmap references non-existent config key: service-ca.crt Sep 06 12:47:06.456491 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:06.456285 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 06 12:47:06.456491 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:06.456292 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs podName:75be41fd-be89-4ca5-9bab-ac64e74295ba nodeName:}" failed. No retries permitted until 2026-09-06 12:47:07.456274803 +0000 UTC m=+37.198645951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs") pod "router-default-659dc76f98-xl7g6" (UID: "75be41fd-be89-4ca5-9bab-ac64e74295ba") : secret "router-metrics-certs-default" not found Sep 06 12:47:06.456491 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:06.456314 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls podName:0c15574a-7aa1-4f24-b884-90ec43811ef2 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:07.45630437 +0000 UTC m=+37.198675503 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-b5zlx" (UID: "0c15574a-7aa1-4f24-b884-90ec43811ef2") : secret "samples-operator-tls" not found Sep 06 12:47:06.456491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:06.456411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:06.456946 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:06.456506 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 06 12:47:06.456946 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:06.456520 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777498f4c-ccqwz: secret "image-registry-tls" not found Sep 06 12:47:06.456946 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:06.456562 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls podName:f220f886-f2be-4f33-b8fc-1d0f3dd3dafd nodeName:}" failed. No retries permitted until 2026-09-06 12:47:07.456550194 +0000 UTC m=+37.198921354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls") pod "image-registry-6777498f4c-ccqwz" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd") : secret "image-registry-tls" not found Sep 06 12:47:07.333905 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.333866 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dhmsm_ff818479-7d03-4a64-8e09-2b094c4011a1/konnectivity-agent/0.log" limit=1024 Sep 06 12:47:07.392879 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.392849 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dhmsm_ff818479-7d03-4a64-8e09-2b094c4011a1/konnectivity-agent/0.log" limit=1024 Sep 06 12:47:07.467792 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.467750 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:07.467963 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.467816 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:07.467963 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.467841 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:07.467963 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.467865 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:07.467963 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.467892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:07.467963 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:07.467910 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 06 12:47:07.467963 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:07.467934 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777498f4c-ccqwz: secret "image-registry-tls" not found Sep 06 12:47:07.467963 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:07.467942 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 06 12:47:07.467963 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:07.467960 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 06 12:47:07.468238 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:07.467980 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 06 12:47:07.468238 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:07.467995 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 06 12:47:07.468238 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:07.468008 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls podName:f220f886-f2be-4f33-b8fc-1d0f3dd3dafd nodeName:}" failed. No retries permitted until 2026-09-06 12:47:09.467976431 +0000 UTC m=+39.210347583 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls") pod "image-registry-6777498f4c-ccqwz" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd") : secret "image-registry-tls" not found Sep 06 12:47:07.468238 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:07.468029 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls podName:0c15574a-7aa1-4f24-b884-90ec43811ef2 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:09.468014531 +0000 UTC m=+39.210385671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-b5zlx" (UID: "0c15574a-7aa1-4f24-b884-90ec43811ef2") : secret "samples-operator-tls" not found Sep 06 12:47:07.468238 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.467919 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-b5zlx\" (UID: \"0c15574a-7aa1-4f24-b884-90ec43811ef2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:07.468238 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:07.468043 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs podName:75be41fd-be89-4ca5-9bab-ac64e74295ba nodeName:}" failed. No retries permitted until 2026-09-06 12:47:09.468037288 +0000 UTC m=+39.210408421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs") pod "router-default-659dc76f98-xl7g6" (UID: "75be41fd-be89-4ca5-9bab-ac64e74295ba") : secret "router-metrics-certs-default" not found Sep 06 12:47:07.468238 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:07.468062 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls podName:c34f36a1-7b51-43f1-a481-e5331b2ee206 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:09.468053334 +0000 UTC m=+39.210424474 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls") pod "dns-default-fgcpf" (UID: "c34f36a1-7b51-43f1-a481-e5331b2ee206") : secret "dns-default-metrics-tls" not found Sep 06 12:47:07.468238 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:07.468081 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle podName:75be41fd-be89-4ca5-9bab-ac64e74295ba nodeName:}" failed. No retries permitted until 2026-09-06 12:47:09.468075846 +0000 UTC m=+39.210446982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle") pod "router-default-659dc76f98-xl7g6" (UID: "75be41fd-be89-4ca5-9bab-ac64e74295ba") : configmap references non-existent config key: service-ca.crt Sep 06 12:47:07.468238 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:07.468096 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls podName:b8dd16fa-c99c-4c68-b472-4927f82d5e5a nodeName:}" failed. No retries permitted until 2026-09-06 12:47:09.46809066 +0000 UTC m=+39.210461792 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9gph2" (UID: "b8dd16fa-c99c-4c68-b472-4927f82d5e5a") : secret "cluster-monitoring-operator-tls" not found Sep 06 12:47:07.585763 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.585733 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-8zhfs"] Sep 06 12:47:07.589421 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:07.589388 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c2987eb_10ff_4823_966e_7e65d410fc56.slice/crio-d648dfc67262f473eb3e3ca23dd84abad68cedea2682b275b71755b28e783981 WatchSource:0}: Error finding container d648dfc67262f473eb3e3ca23dd84abad68cedea2682b275b71755b28e783981: Status 404 returned error can't find the container with id d648dfc67262f473eb3e3ca23dd84abad68cedea2682b275b71755b28e783981 Sep 06 12:47:07.591929 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.591904 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght"] Sep 06 12:47:07.595730 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.595703 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-2vfhs"] Sep 06 12:47:07.598301 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.598278 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-996bm"] Sep 06 12:47:07.599478 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:07.599234 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6850db_dfc3_47f9_b91b_54757857d601.slice/crio-622c588c5af08afd254e4d7eb84742ff7695449b51725076f41168e5feb4a8ed WatchSource:0}: Error finding container 622c588c5af08afd254e4d7eb84742ff7695449b51725076f41168e5feb4a8ed: Status 404 returned error can't find the container with id 622c588c5af08afd254e4d7eb84742ff7695449b51725076f41168e5feb4a8ed Sep 06 12:47:07.599547 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.599506 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl"] Sep 06 12:47:07.599745 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:07.599662 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8750b827_635a_4805_817c_45a970d9fc81.slice/crio-d5317dabb87bdb19840928da5774478fc4884188252df32174f898abb4fcaaf9 WatchSource:0}: Error finding container d5317dabb87bdb19840928da5774478fc4884188252df32174f898abb4fcaaf9: Status 404 returned error can't find the container with id d5317dabb87bdb19840928da5774478fc4884188252df32174f898abb4fcaaf9 Sep 06 12:47:07.601859 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:07.601827 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4b28cb_5070_41be_81d6_bf4965e44af1.slice/crio-87eecc8c4064fb4c66701c9402abea874b7c76e8e3f731a7b9199a14d666f46a WatchSource:0}: Error finding container 87eecc8c4064fb4c66701c9402abea874b7c76e8e3f731a7b9199a14d666f46a: Status 404 returned error can't find the container with id 87eecc8c4064fb4c66701c9402abea874b7c76e8e3f731a7b9199a14d666f46a Sep 06 12:47:07.604872 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:07.604854 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb93d75ad_6183_460c_9652_c502a31ba231.slice/crio-b7af0c59ffb61be29090bc5b21d428280d0d79260255fdf47518c15ffb558803 WatchSource:0}: Error finding container b7af0c59ffb61be29090bc5b21d428280d0d79260255fdf47518c15ffb558803: Status 404 returned error can't find the container with id b7af0c59ffb61be29090bc5b21d428280d0d79260255fdf47518c15ffb558803 Sep 06 12:47:07.609151 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.609102 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-wccw5"] Sep 06 12:47:07.609925 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.609891 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h8tgn"] Sep 06 12:47:07.987114 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.987076 2580 generic.go:356] "Generic (PLEG): container finished" podID="55fb0a7b-617b-477c-ae3a-2c6cf1b33480" containerID="9d9e8543e99fa2766bfb906d3baff66ae8eecacf51a0a58e78a00fb0bcbbf256" exitCode=0 Sep 06 12:47:07.987315 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.987157 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmxfr" event={"ID":"55fb0a7b-617b-477c-ae3a-2c6cf1b33480","Type":"ContainerDied","Data":"9d9e8543e99fa2766bfb906d3baff66ae8eecacf51a0a58e78a00fb0bcbbf256"} Sep 06 12:47:07.988517 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.988489 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-8zhfs" event={"ID":"9c2987eb-10ff-4823-966e-7e65d410fc56","Type":"ContainerStarted","Data":"d648dfc67262f473eb3e3ca23dd84abad68cedea2682b275b71755b28e783981"} Sep 06 12:47:07.989874 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.989849 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" event={"ID":"b93d75ad-6183-460c-9652-c502a31ba231","Type":"ContainerStarted","Data":"b7af0c59ffb61be29090bc5b21d428280d0d79260255fdf47518c15ffb558803"} Sep 06 12:47:07.994142 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.994085 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h8tgn" event={"ID":"f09a5d91-5be0-4cff-8388-d3f33dfb0391","Type":"ContainerStarted","Data":"65287b80acb7848b99bf6889cd23390af01e34bb3e2a21865d488af8fa429cd4"} Sep 06 12:47:07.996005 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.995970 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" event={"ID":"fc4b28cb-5070-41be-81d6-bf4965e44af1","Type":"ContainerStarted","Data":"87eecc8c4064fb4c66701c9402abea874b7c76e8e3f731a7b9199a14d666f46a"} Sep 06 12:47:07.997288 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.997248 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" event={"ID":"8750b827-635a-4805-817c-45a970d9fc81","Type":"ContainerStarted","Data":"d5317dabb87bdb19840928da5774478fc4884188252df32174f898abb4fcaaf9"} Sep 06 12:47:07.998477 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.998452 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-wccw5" event={"ID":"73ed942b-28b8-492f-ac72-65e2dfa64b25","Type":"ContainerStarted","Data":"454f6247db12a950f0df430234ab9980a197e1bb5786491b758b6c35fa773254"} Sep 06 12:47:07.999627 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:07.999607 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" event={"ID":"ab6850db-dfc3-47f9-b91b-54757857d601","Type":"ContainerStarted","Data":"622c588c5af08afd254e4d7eb84742ff7695449b51725076f41168e5feb4a8ed"} Sep 06 12:47:09.013688 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:09.013387 2580 generic.go:356] "Generic (PLEG): container finished" podID="55fb0a7b-617b-477c-ae3a-2c6cf1b33480" containerID="dcad05430bdee3ec0a9ef26d300fe8516052fcb6c76e55bf3245dcc71bf10f83" exitCode=0 Sep 06 12:47:09.013688 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:09.013684 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmxfr" event={"ID":"55fb0a7b-617b-477c-ae3a-2c6cf1b33480","Type":"ContainerDied","Data":"dcad05430bdee3ec0a9ef26d300fe8516052fcb6c76e55bf3245dcc71bf10f83"} Sep 06 12:47:09.487323 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:09.486811 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:09.487463 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:09.487419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:09.487521 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:09.487486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:09.487590 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:09.487526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:09.487590 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:09.487565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:09.487814 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:09.487792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-b5zlx\" (UID: \"0c15574a-7aa1-4f24-b884-90ec43811ef2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:09.495571 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:09.492840 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 06 12:47:09.495571 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:09.492845 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 06 12:47:09.495571 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:09.492906 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle podName:75be41fd-be89-4ca5-9bab-ac64e74295ba nodeName:}" failed. No retries permitted until 2026-09-06 12:47:13.492875546 +0000 UTC m=+43.235246693 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle") pod "router-default-659dc76f98-xl7g6" (UID: "75be41fd-be89-4ca5-9bab-ac64e74295ba") : configmap references non-existent config key: service-ca.crt Sep 06 12:47:09.495571 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:09.492945 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs podName:75be41fd-be89-4ca5-9bab-ac64e74295ba nodeName:}" failed. No retries permitted until 2026-09-06 12:47:13.49292133 +0000 UTC m=+43.235292470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs") pod "router-default-659dc76f98-xl7g6" (UID: "75be41fd-be89-4ca5-9bab-ac64e74295ba") : secret "router-metrics-certs-default" not found Sep 06 12:47:09.495571 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:09.492947 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 06 12:47:09.495571 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:09.492965 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 06 12:47:09.495571 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:09.492966 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls podName:b8dd16fa-c99c-4c68-b472-4927f82d5e5a nodeName:}" failed. No retries permitted until 2026-09-06 12:47:13.492955067 +0000 UTC m=+43.235326211 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9gph2" (UID: "b8dd16fa-c99c-4c68-b472-4927f82d5e5a") : secret "cluster-monitoring-operator-tls" not found Sep 06 12:47:09.495571 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:09.493038 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls podName:0c15574a-7aa1-4f24-b884-90ec43811ef2 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:13.493023298 +0000 UTC m=+43.235394431 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-b5zlx" (UID: "0c15574a-7aa1-4f24-b884-90ec43811ef2") : secret "samples-operator-tls" not found Sep 06 12:47:09.495571 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:09.493042 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 06 12:47:09.495571 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:09.493055 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777498f4c-ccqwz: secret "image-registry-tls" not found Sep 06 12:47:09.495571 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:09.493055 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls podName:c34f36a1-7b51-43f1-a481-e5331b2ee206 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:13.493045513 +0000 UTC m=+43.235416647 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls") pod "dns-default-fgcpf" (UID: "c34f36a1-7b51-43f1-a481-e5331b2ee206") : secret "dns-default-metrics-tls" not found Sep 06 12:47:09.495571 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:09.493094 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls podName:f220f886-f2be-4f33-b8fc-1d0f3dd3dafd nodeName:}" failed. No retries permitted until 2026-09-06 12:47:13.493083204 +0000 UTC m=+43.235454339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls") pod "image-registry-6777498f4c-ccqwz" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd") : secret "image-registry-tls" not found Sep 06 12:47:10.024840 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:10.024751 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmxfr" event={"ID":"55fb0a7b-617b-477c-ae3a-2c6cf1b33480","Type":"ContainerStarted","Data":"593d77e2bbe590d98b84a1a95a5a8c0a86db8a1a25a329f07c7c7dee25113162"} Sep 06 12:47:10.046347 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:10.046287 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gmxfr" podStartSLOduration=8.537244868 podStartE2EDuration="39.046251607s" totalImagesPullingTime="30.509006739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:10.045509001 +0000 UTC m=+39.787880159" watchObservedRunningTime="2026-09-06 12:47:10.046251607 +0000 UTC m=+39.788622764" Sep 06 12:47:12.482142 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:12.482103 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-d2c2j"] Sep 06 12:47:12.512394 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:12.512364 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-d2c2j"] Sep 06 12:47:12.512572 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:12.512478 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:12.514975 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:12.514762 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 06 12:47:12.514975 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:12.514786 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 06 12:47:12.514975 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:12.514765 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xjzzn\"" Sep 06 12:47:12.618011 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:12.617975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c28de569-a5a3-46dc-b764-7244ecb5b380-nginx-conf\") pod \"networking-console-plugin-79977f9844-d2c2j\" (UID: \"c28de569-a5a3-46dc-b764-7244ecb5b380\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:12.618168 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:12.618019 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d2c2j\" (UID: \"c28de569-a5a3-46dc-b764-7244ecb5b380\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:12.718744 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:12.718704 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c28de569-a5a3-46dc-b764-7244ecb5b380-nginx-conf\") pod \"networking-console-plugin-79977f9844-d2c2j\" (UID: \"c28de569-a5a3-46dc-b764-7244ecb5b380\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:12.718894 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:12.718755 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d2c2j\" (UID: \"c28de569-a5a3-46dc-b764-7244ecb5b380\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:12.718954 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:12.718894 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 06 12:47:12.718998 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:12.718967 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert podName:c28de569-a5a3-46dc-b764-7244ecb5b380 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:13.218949993 +0000 UTC m=+42.961321130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-d2c2j" (UID: "c28de569-a5a3-46dc-b764-7244ecb5b380") : secret "networking-console-plugin-cert" not found Sep 06 12:47:12.719606 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:12.719580 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c28de569-a5a3-46dc-b764-7244ecb5b380-nginx-conf\") pod \"networking-console-plugin-79977f9844-d2c2j\" (UID: \"c28de569-a5a3-46dc-b764-7244ecb5b380\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:13.223851 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:13.223811 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d2c2j\" (UID: \"c28de569-a5a3-46dc-b764-7244ecb5b380\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:13.224045 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.223964 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 06 12:47:13.224045 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.224033 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert podName:c28de569-a5a3-46dc-b764-7244ecb5b380 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:14.224016227 +0000 UTC m=+43.966387378 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-d2c2j" (UID: "c28de569-a5a3-46dc-b764-7244ecb5b380") : secret "networking-console-plugin-cert" not found Sep 06 12:47:13.526888 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:13.526793 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:13.526888 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:13.526865 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:13.526904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:13.526934 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:13.526963 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-b5zlx\" (UID: \"0c15574a-7aa1-4f24-b884-90ec43811ef2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:13.527057 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.526960 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.527100 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle podName:75be41fd-be89-4ca5-9bab-ac64e74295ba nodeName:}" failed. No retries permitted until 2026-09-06 12:47:21.527079391 +0000 UTC m=+51.269450547 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle") pod "router-default-659dc76f98-xl7g6" (UID: "75be41fd-be89-4ca5-9bab-ac64e74295ba") : configmap references non-existent config key: service-ca.crt Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.527106 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.527136 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.527146 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777498f4c-ccqwz: secret "image-registry-tls" not found Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.527022 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.527100 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.527167 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs podName:75be41fd-be89-4ca5-9bab-ac64e74295ba nodeName:}" failed. No retries permitted until 2026-09-06 12:47:21.527150761 +0000 UTC m=+51.269521903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs") pod "router-default-659dc76f98-xl7g6" (UID: "75be41fd-be89-4ca5-9bab-ac64e74295ba") : secret "router-metrics-certs-default" not found Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.527318 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls podName:0c15574a-7aa1-4f24-b884-90ec43811ef2 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:21.527286864 +0000 UTC m=+51.269657997 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-b5zlx" (UID: "0c15574a-7aa1-4f24-b884-90ec43811ef2") : secret "samples-operator-tls" not found Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.527342 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls podName:f220f886-f2be-4f33-b8fc-1d0f3dd3dafd nodeName:}" failed. No retries permitted until 2026-09-06 12:47:21.527334831 +0000 UTC m=+51.269705964 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls") pod "image-registry-6777498f4c-ccqwz" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd") : secret "image-registry-tls" not found Sep 06 12:47:13.527403 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.527353 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls podName:b8dd16fa-c99c-4c68-b472-4927f82d5e5a nodeName:}" failed. No retries permitted until 2026-09-06 12:47:21.527347856 +0000 UTC m=+51.269718989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9gph2" (UID: "b8dd16fa-c99c-4c68-b472-4927f82d5e5a") : secret "cluster-monitoring-operator-tls" not found Sep 06 12:47:13.528074 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:13.527364 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls podName:c34f36a1-7b51-43f1-a481-e5331b2ee206 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:21.527359091 +0000 UTC m=+51.269730223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls") pod "dns-default-fgcpf" (UID: "c34f36a1-7b51-43f1-a481-e5331b2ee206") : secret "dns-default-metrics-tls" not found Sep 06 12:47:14.232957 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:14.232914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d2c2j\" (UID: \"c28de569-a5a3-46dc-b764-7244ecb5b380\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:14.233112 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:14.233077 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 06 12:47:14.233153 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:14.233145 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert podName:c28de569-a5a3-46dc-b764-7244ecb5b380 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:16.233128789 +0000 UTC m=+45.975499925 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-d2c2j" (UID: "c28de569-a5a3-46dc-b764-7244ecb5b380") : secret "networking-console-plugin-cert" not found Sep 06 12:47:16.049033 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.048183 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" event={"ID":"fc4b28cb-5070-41be-81d6-bf4965e44af1","Type":"ContainerStarted","Data":"2da59ecb063ec6bfe8ef4b59adbcf3692931bda914e9fc42adcb5430e7bf2b7e"} Sep 06 12:47:16.050934 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.050909 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/0.log" Sep 06 12:47:16.051061 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.050953 2580 generic.go:356] "Generic (PLEG): container finished" podID="8750b827-635a-4805-817c-45a970d9fc81" containerID="d40b3eb3bca5cc404c7e6dd4525f697d8d43224f2e933c9b3dae7717cd34bb8b" exitCode=255 Sep 06 12:47:16.051061 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.051032 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" event={"ID":"8750b827-635a-4805-817c-45a970d9fc81","Type":"ContainerDied","Data":"d40b3eb3bca5cc404c7e6dd4525f697d8d43224f2e933c9b3dae7717cd34bb8b"} Sep 06 12:47:16.051247 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.051227 2580 scope.go:117] "RemoveContainer" containerID="d40b3eb3bca5cc404c7e6dd4525f697d8d43224f2e933c9b3dae7717cd34bb8b" Sep 06 12:47:16.053671 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.053647 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-wccw5" event={"ID":"73ed942b-28b8-492f-ac72-65e2dfa64b25","Type":"ContainerStarted","Data":"1d06696d4e9af021fe655c541485a92b3de4dab3b1780140d0a4e4a7b8bf43e5"} Sep 06 12:47:16.055504 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.055482 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" event={"ID":"ab6850db-dfc3-47f9-b91b-54757857d601","Type":"ContainerStarted","Data":"af4cdd27a37f614803e0e4e4fd3f3fb351007f594329c33bbf4c31105014d8c6"} Sep 06 12:47:16.057049 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.057026 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-8zhfs" event={"ID":"9c2987eb-10ff-4823-966e-7e65d410fc56","Type":"ContainerStarted","Data":"7d3a1c88ea4ec8c17e251662ee9ea55df2227d1e8fe41c5be5e6f94bdf9bfc4e"} Sep 06 12:47:16.058994 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.058974 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" event={"ID":"b93d75ad-6183-460c-9652-c502a31ba231","Type":"ContainerStarted","Data":"b00d14d544f731c010c7664833f766cd2f2b3d8d972f1d7379c863fd7e366227"} Sep 06 12:47:16.060885 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.060834 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h8tgn" event={"ID":"f09a5d91-5be0-4cff-8388-d3f33dfb0391","Type":"ContainerStarted","Data":"13a4834f99b4a9ab718d6b41f74910e16a270afaad2494b53a1816d45f670edd"} Sep 06 12:47:16.064834 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.064789 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" podStartSLOduration=24.409178025 podStartE2EDuration="32.064773705s" totalImagesPullingTime="7.65559568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:16.063686517 +0000 UTC m=+45.806057672" watchObservedRunningTime="2026-09-06 12:47:16.064773705 +0000 UTC m=+45.807144853" Sep 06 12:47:16.078486 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.077568 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-8zhfs" podStartSLOduration=31.85113038 podStartE2EDuration="39.077551377s" totalImagesPullingTime="7.226420997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:16.076930236 +0000 UTC m=+45.819301392" watchObservedRunningTime="2026-09-06 12:47:16.077551377 +0000 UTC m=+45.819922531" Sep 06 12:47:16.096399 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.096352 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" podStartSLOduration=26.431000808 podStartE2EDuration="34.096252954s" totalImagesPullingTime="7.665252146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:16.09483757 +0000 UTC m=+45.837208721" watchObservedRunningTime="2026-09-06 12:47:16.096252954 +0000 UTC m=+45.838624114" Sep 06 12:47:16.110959 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.110921 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-h8tgn" podStartSLOduration=3.466848572 podStartE2EDuration="11.110904701s" totalImagesPullingTime="7.644056129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:47:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:16.110404564 +0000 UTC m=+45.852775720" watchObservedRunningTime="2026-09-06 12:47:16.110904701 +0000 UTC m=+45.853275858" Sep 06 12:47:16.149074 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.149007 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" podStartSLOduration=31.496494105 podStartE2EDuration="39.148985356s" totalImagesPullingTime="7.652491251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:16.147777598 +0000 UTC m=+45.890148754" watchObservedRunningTime="2026-09-06 12:47:16.148985356 +0000 UTC m=+45.891356515" Sep 06 12:47:16.250063 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.250028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d2c2j\" (UID: \"c28de569-a5a3-46dc-b764-7244ecb5b380\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:16.250215 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:16.250194 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 06 12:47:16.250302 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:16.250290 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert podName:c28de569-a5a3-46dc-b764-7244ecb5b380 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:20.250257577 +0000 UTC m=+49.992628716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-d2c2j" (UID: "c28de569-a5a3-46dc-b764-7244ecb5b380") : secret "networking-console-plugin-cert" not found Sep 06 12:47:16.934693 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.934594 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-wccw5" podStartSLOduration=24.290219971 podStartE2EDuration="31.934577811s" totalImagesPullingTime="7.64435784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:16.164131764 +0000 UTC m=+45.906502911" watchObservedRunningTime="2026-09-06 12:47:16.934577811 +0000 UTC m=+46.676949043" Sep 06 12:47:16.934915 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.934761 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-799wl"] Sep 06 12:47:16.946722 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.946699 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-799wl"] Sep 06 12:47:16.946858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.946818 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-799wl" Sep 06 12:47:16.948978 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.948952 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-g9pt7\"" Sep 06 12:47:16.949133 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.949111 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 06 12:47:16.949745 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:16.949726 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 06 12:47:17.058275 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.058239 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wczwd\" (UniqueName: \"kubernetes.io/projected/0e03a1d0-49d0-4899-86ce-779bf37dd7a6-kube-api-access-wczwd\") pod \"migrator-fdbb8fdff-799wl\" (UID: \"0e03a1d0-49d0-4899-86ce-779bf37dd7a6\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-799wl" Sep 06 12:47:17.065524 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.065503 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 12:47:17.065952 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.065938 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/0.log" Sep 06 12:47:17.066013 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.065971 2580 generic.go:356] "Generic (PLEG): container finished" podID="8750b827-635a-4805-817c-45a970d9fc81" containerID="512543a62434237f32385bc61fbaf544f11382fda1c4700a39d440e7a28b749d" exitCode=255 Sep 06 12:47:17.066063 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.066051 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" event={"ID":"8750b827-635a-4805-817c-45a970d9fc81","Type":"ContainerDied","Data":"512543a62434237f32385bc61fbaf544f11382fda1c4700a39d440e7a28b749d"} Sep 06 12:47:17.066114 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.066097 2580 scope.go:117] "RemoveContainer" containerID="d40b3eb3bca5cc404c7e6dd4525f697d8d43224f2e933c9b3dae7717cd34bb8b" Sep 06 12:47:17.066378 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.066351 2580 scope.go:117] "RemoveContainer" containerID="512543a62434237f32385bc61fbaf544f11382fda1c4700a39d440e7a28b749d" Sep 06 12:47:17.066564 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:17.066541 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-2vfhs_openshift-console-operator(8750b827-635a-4805-817c-45a970d9fc81)\"" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" podUID="8750b827-635a-4805-817c-45a970d9fc81" Sep 06 12:47:17.160158 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.159550 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wczwd\" (UniqueName: \"kubernetes.io/projected/0e03a1d0-49d0-4899-86ce-779bf37dd7a6-kube-api-access-wczwd\") pod \"migrator-fdbb8fdff-799wl\" (UID: \"0e03a1d0-49d0-4899-86ce-779bf37dd7a6\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-799wl" Sep 06 12:47:17.169063 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.169029 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wczwd\" (UniqueName: \"kubernetes.io/projected/0e03a1d0-49d0-4899-86ce-779bf37dd7a6-kube-api-access-wczwd\") pod \"migrator-fdbb8fdff-799wl\" (UID: \"0e03a1d0-49d0-4899-86ce-779bf37dd7a6\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-799wl" Sep 06 12:47:17.257898 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.257826 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-799wl" Sep 06 12:47:17.494772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.494736 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-462kn"] Sep 06 12:47:17.527369 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.527291 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-462kn"] Sep 06 12:47:17.527521 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.527471 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.529742 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.529672 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 06 12:47:17.529742 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.529714 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mb5b8\"" Sep 06 12:47:17.529931 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.529813 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 06 12:47:17.558815 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.558787 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-799wl"] Sep 06 12:47:17.561909 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:17.561882 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e03a1d0_49d0_4899_86ce_779bf37dd7a6.slice/crio-017408c35df2831b1c21e6c8f8e416278661fa9e5f9241405b53bc4939aedba2 WatchSource:0}: Error finding container 017408c35df2831b1c21e6c8f8e416278661fa9e5f9241405b53bc4939aedba2: Status 404 returned error can't find the container with id 017408c35df2831b1c21e6c8f8e416278661fa9e5f9241405b53bc4939aedba2 Sep 06 12:47:17.562161 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.562142 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.562223 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.562177 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.562310 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.562294 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d654b09-beec-407d-b4b7-cc802e221d43-data-volume\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.562475 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.562448 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xx4h\" (UniqueName: \"kubernetes.io/projected/5d654b09-beec-407d-b4b7-cc802e221d43-kube-api-access-5xx4h\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.562577 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.562555 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5d654b09-beec-407d-b4b7-cc802e221d43-crio-socket\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.663298 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.663258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xx4h\" (UniqueName: \"kubernetes.io/projected/5d654b09-beec-407d-b4b7-cc802e221d43-kube-api-access-5xx4h\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.663420 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.663314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5d654b09-beec-407d-b4b7-cc802e221d43-crio-socket\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.663420 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.663355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.663420 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.663382 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.663590 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.663432 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d654b09-beec-407d-b4b7-cc802e221d43-data-volume\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.663590 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:17.663501 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 06 12:47:17.663590 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:17.663526 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 06 12:47:17.663590 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.663543 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5d654b09-beec-407d-b4b7-cc802e221d43-crio-socket\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.663590 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:17.663589 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm podName:5d654b09-beec-407d-b4b7-cc802e221d43 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:18.163567841 +0000 UTC m=+47.905938989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-462kn" (UID: "5d654b09-beec-407d-b4b7-cc802e221d43") : configmap "kube-rbac-proxy" not found Sep 06 12:47:17.663801 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:17.663611 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls podName:5d654b09-beec-407d-b4b7-cc802e221d43 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:18.163601429 +0000 UTC m=+47.905972570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-462kn" (UID: "5d654b09-beec-407d-b4b7-cc802e221d43") : secret "insights-runtime-extractor-tls" not found Sep 06 12:47:17.663801 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.663717 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d654b09-beec-407d-b4b7-cc802e221d43-data-volume\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:17.671196 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:17.671176 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xx4h\" (UniqueName: \"kubernetes.io/projected/5d654b09-beec-407d-b4b7-cc802e221d43-kube-api-access-5xx4h\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:18.071074 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:18.071044 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 12:47:18.071539 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:18.071447 2580 scope.go:117] "RemoveContainer" containerID="512543a62434237f32385bc61fbaf544f11382fda1c4700a39d440e7a28b749d" Sep 06 12:47:18.071648 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:18.071622 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-2vfhs_openshift-console-operator(8750b827-635a-4805-817c-45a970d9fc81)\"" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" podUID="8750b827-635a-4805-817c-45a970d9fc81" Sep 06 12:47:18.072432 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:18.072406 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-799wl" event={"ID":"0e03a1d0-49d0-4899-86ce-779bf37dd7a6","Type":"ContainerStarted","Data":"017408c35df2831b1c21e6c8f8e416278661fa9e5f9241405b53bc4939aedba2"} Sep 06 12:47:18.168513 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:18.168483 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:18.168691 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:18.168524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:18.168691 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:18.168598 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 06 12:47:18.168691 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:18.168636 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 06 12:47:18.168691 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:18.168658 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm podName:5d654b09-beec-407d-b4b7-cc802e221d43 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:19.168639599 +0000 UTC m=+48.911010732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-462kn" (UID: "5d654b09-beec-407d-b4b7-cc802e221d43") : configmap "kube-rbac-proxy" not found Sep 06 12:47:18.168691 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:18.168675 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls podName:5d654b09-beec-407d-b4b7-cc802e221d43 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:19.168664881 +0000 UTC m=+48.911036019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-462kn" (UID: "5d654b09-beec-407d-b4b7-cc802e221d43") : secret "insights-runtime-extractor-tls" not found Sep 06 12:47:19.103646 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.103609 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-ssmnt"] Sep 06 12:47:19.120005 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.119968 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-ssmnt"] Sep 06 12:47:19.120141 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.120086 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-ssmnt" Sep 06 12:47:19.122561 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.122536 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 06 12:47:19.122561 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.122557 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 06 12:47:19.122724 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.122540 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 06 12:47:19.123290 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.123253 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-lml5p\"" Sep 06 12:47:19.123380 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.123342 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 06 12:47:19.179015 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.178983 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:19.179015 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.179019 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:19.179242 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.179039 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f5a03ade-d5e5-48dd-8cd1-c0a437882779-signing-key\") pod \"service-ca-75d99c484-ssmnt\" (UID: \"f5a03ade-d5e5-48dd-8cd1-c0a437882779\") " pod="openshift-service-ca/service-ca-75d99c484-ssmnt" Sep 06 12:47:19.179242 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.179069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f5a03ade-d5e5-48dd-8cd1-c0a437882779-signing-cabundle\") pod \"service-ca-75d99c484-ssmnt\" (UID: \"f5a03ade-d5e5-48dd-8cd1-c0a437882779\") " pod="openshift-service-ca/service-ca-75d99c484-ssmnt" Sep 06 12:47:19.179242 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:19.179105 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 06 12:47:19.179242 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:19.179138 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 06 12:47:19.179242 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.179170 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v8sx\" (UniqueName: \"kubernetes.io/projected/f5a03ade-d5e5-48dd-8cd1-c0a437882779-kube-api-access-2v8sx\") pod \"service-ca-75d99c484-ssmnt\" (UID: \"f5a03ade-d5e5-48dd-8cd1-c0a437882779\") " pod="openshift-service-ca/service-ca-75d99c484-ssmnt" Sep 06 12:47:19.179242 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:19.179217 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm podName:5d654b09-beec-407d-b4b7-cc802e221d43 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:21.179194217 +0000 UTC m=+50.921565365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-462kn" (UID: "5d654b09-beec-407d-b4b7-cc802e221d43") : configmap "kube-rbac-proxy" not found Sep 06 12:47:19.179556 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:19.179291 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls podName:5d654b09-beec-407d-b4b7-cc802e221d43 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:21.179278134 +0000 UTC m=+50.921649281 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-462kn" (UID: "5d654b09-beec-407d-b4b7-cc802e221d43") : secret "insights-runtime-extractor-tls" not found Sep 06 12:47:19.280090 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.280063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v8sx\" (UniqueName: \"kubernetes.io/projected/f5a03ade-d5e5-48dd-8cd1-c0a437882779-kube-api-access-2v8sx\") pod \"service-ca-75d99c484-ssmnt\" (UID: \"f5a03ade-d5e5-48dd-8cd1-c0a437882779\") " pod="openshift-service-ca/service-ca-75d99c484-ssmnt" Sep 06 12:47:19.280415 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.280392 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f5a03ade-d5e5-48dd-8cd1-c0a437882779-signing-key\") pod \"service-ca-75d99c484-ssmnt\" (UID: \"f5a03ade-d5e5-48dd-8cd1-c0a437882779\") " pod="openshift-service-ca/service-ca-75d99c484-ssmnt" Sep 06 12:47:19.280468 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.280457 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f5a03ade-d5e5-48dd-8cd1-c0a437882779-signing-cabundle\") pod \"service-ca-75d99c484-ssmnt\" (UID: \"f5a03ade-d5e5-48dd-8cd1-c0a437882779\") " pod="openshift-service-ca/service-ca-75d99c484-ssmnt" Sep 06 12:47:19.281151 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.281122 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f5a03ade-d5e5-48dd-8cd1-c0a437882779-signing-cabundle\") pod \"service-ca-75d99c484-ssmnt\" (UID: \"f5a03ade-d5e5-48dd-8cd1-c0a437882779\") " pod="openshift-service-ca/service-ca-75d99c484-ssmnt" Sep 06 12:47:19.282773 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.282747 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f5a03ade-d5e5-48dd-8cd1-c0a437882779-signing-key\") pod \"service-ca-75d99c484-ssmnt\" (UID: \"f5a03ade-d5e5-48dd-8cd1-c0a437882779\") " pod="openshift-service-ca/service-ca-75d99c484-ssmnt" Sep 06 12:47:19.283691 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.283676 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-89gcv_3568c869-d959-469d-be7e-c73faa22d4d9/dns-node-resolver/0.log" Sep 06 12:47:19.288385 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.288364 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v8sx\" (UniqueName: \"kubernetes.io/projected/f5a03ade-d5e5-48dd-8cd1-c0a437882779-kube-api-access-2v8sx\") pod \"service-ca-75d99c484-ssmnt\" (UID: \"f5a03ade-d5e5-48dd-8cd1-c0a437882779\") " pod="openshift-service-ca/service-ca-75d99c484-ssmnt" Sep 06 12:47:19.429076 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.429053 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-ssmnt" Sep 06 12:47:19.733544 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:19.733472 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-ssmnt"] Sep 06 12:47:19.736827 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:19.736802 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5a03ade_d5e5_48dd_8cd1_c0a437882779.slice/crio-8690e01f373508f937c43ccb0454188a1cf4042ccd45a1a5bd3e2af1a179d371 WatchSource:0}: Error finding container 8690e01f373508f937c43ccb0454188a1cf4042ccd45a1a5bd3e2af1a179d371: Status 404 returned error can't find the container with id 8690e01f373508f937c43ccb0454188a1cf4042ccd45a1a5bd3e2af1a179d371 Sep 06 12:47:20.086554 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:20.086516 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-ssmnt" event={"ID":"f5a03ade-d5e5-48dd-8cd1-c0a437882779","Type":"ContainerStarted","Data":"57f0912b5825a269e27925b9eb98057981fa8a935f652a102144c051d93c7b90"} Sep 06 12:47:20.086744 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:20.086604 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-ssmnt" event={"ID":"f5a03ade-d5e5-48dd-8cd1-c0a437882779","Type":"ContainerStarted","Data":"8690e01f373508f937c43ccb0454188a1cf4042ccd45a1a5bd3e2af1a179d371"} Sep 06 12:47:20.088364 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:20.088338 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-799wl" event={"ID":"0e03a1d0-49d0-4899-86ce-779bf37dd7a6","Type":"ContainerStarted","Data":"17a7383009c04ffb6f4e8f986c3c0b7a485724ac099ca12f9d45b095ada36e66"} Sep 06 12:47:20.088481 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:20.088380 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-799wl" event={"ID":"0e03a1d0-49d0-4899-86ce-779bf37dd7a6","Type":"ContainerStarted","Data":"3a823764b722f0bf830efcb7b3dc1498d0be299022e8f6a9ee83f9a61a30ba2e"} Sep 06 12:47:20.107897 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:20.107851 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-ssmnt" podStartSLOduration=1.107838759 podStartE2EDuration="1.107838759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:47:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:20.106862392 +0000 UTC m=+49.849233546" watchObservedRunningTime="2026-09-06 12:47:20.107838759 +0000 UTC m=+49.850209914" Sep 06 12:47:20.136886 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:20.136838 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-799wl" podStartSLOduration=2.467105559 podStartE2EDuration="4.136823304s" totalImagesPullingTime="1.669717745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:47:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:20.136045604 +0000 UTC m=+49.878416761" watchObservedRunningTime="2026-09-06 12:47:20.136823304 +0000 UTC m=+49.879194459" Sep 06 12:47:20.287971 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:20.287938 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d2c2j\" (UID: \"c28de569-a5a3-46dc-b764-7244ecb5b380\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:20.288121 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:20.288094 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 06 12:47:20.288186 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:20.288161 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert podName:c28de569-a5a3-46dc-b764-7244ecb5b380 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:28.288146046 +0000 UTC m=+58.030517179 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-d2c2j" (UID: "c28de569-a5a3-46dc-b764-7244ecb5b380") : secret "networking-console-plugin-cert" not found Sep 06 12:47:20.464882 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:20.464813 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qrd8m_5c30ce1d-31d0-4c59-b917-59e79bc7093f/node-ca/0.log" Sep 06 12:47:21.067589 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:21.067558 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h8tgn_f09a5d91-5be0-4cff-8388-d3f33dfb0391/serve-healthcheck-canary/0.log" Sep 06 12:47:21.195865 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:21.195824 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:21.196341 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:21.195882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:21.196730 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.196714 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 06 12:47:21.196870 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.196860 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm podName:5d654b09-beec-407d-b4b7-cc802e221d43 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:25.196840258 +0000 UTC m=+54.939211407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-462kn" (UID: "5d654b09-beec-407d-b4b7-cc802e221d43") : configmap "kube-rbac-proxy" not found Sep 06 12:47:21.197002 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.196982 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 06 12:47:21.197056 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.197041 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls podName:5d654b09-beec-407d-b4b7-cc802e221d43 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:25.197024574 +0000 UTC m=+54.939395714 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-462kn" (UID: "5d654b09-beec-407d-b4b7-cc802e221d43") : secret "insights-runtime-extractor-tls" not found Sep 06 12:47:21.262910 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:21.262874 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-799wl_0e03a1d0-49d0-4899-86ce-779bf37dd7a6/migrator/0.log" Sep 06 12:47:21.471741 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:21.471665 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-799wl_0e03a1d0-49d0-4899-86ce-779bf37dd7a6/graceful-termination/0.log" Sep 06 12:47:21.600111 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:21.600075 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:21.600332 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:21.600137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:21.600332 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:21.600170 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:21.600332 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.600297 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 06 12:47:21.600332 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:21.600313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:21.600563 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.600363 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls podName:b8dd16fa-c99c-4c68-b472-4927f82d5e5a nodeName:}" failed. No retries permitted until 2026-09-06 12:47:37.600342231 +0000 UTC m=+67.342713365 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9gph2" (UID: "b8dd16fa-c99c-4c68-b472-4927f82d5e5a") : secret "cluster-monitoring-operator-tls" not found Sep 06 12:47:21.600563 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:21.600397 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-b5zlx\" (UID: \"0c15574a-7aa1-4f24-b884-90ec43811ef2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:21.600563 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:21.600537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:21.600716 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.600665 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 06 12:47:21.600716 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.600678 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777498f4c-ccqwz: secret "image-registry-tls" not found Sep 06 12:47:21.600815 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.600725 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle podName:75be41fd-be89-4ca5-9bab-ac64e74295ba nodeName:}" failed. No retries permitted until 2026-09-06 12:47:37.600708961 +0000 UTC m=+67.343080097 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle") pod "router-default-659dc76f98-xl7g6" (UID: "75be41fd-be89-4ca5-9bab-ac64e74295ba") : configmap references non-existent config key: service-ca.crt Sep 06 12:47:21.600815 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.600743 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls podName:f220f886-f2be-4f33-b8fc-1d0f3dd3dafd nodeName:}" failed. No retries permitted until 2026-09-06 12:47:37.600733982 +0000 UTC m=+67.343105117 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls") pod "image-registry-6777498f4c-ccqwz" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd") : secret "image-registry-tls" not found Sep 06 12:47:21.600815 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.600772 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 06 12:47:21.600815 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.600808 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls podName:0c15574a-7aa1-4f24-b884-90ec43811ef2 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:37.600795368 +0000 UTC m=+67.343166504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-b5zlx" (UID: "0c15574a-7aa1-4f24-b884-90ec43811ef2") : secret "samples-operator-tls" not found Sep 06 12:47:21.601030 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.600871 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 06 12:47:21.601030 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.600907 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls podName:c34f36a1-7b51-43f1-a481-e5331b2ee206 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:37.600893909 +0000 UTC m=+67.343265060 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls") pod "dns-default-fgcpf" (UID: "c34f36a1-7b51-43f1-a481-e5331b2ee206") : secret "dns-default-metrics-tls" not found Sep 06 12:47:21.601030 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.600978 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 06 12:47:21.601030 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:21.601018 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs podName:75be41fd-be89-4ca5-9bab-ac64e74295ba nodeName:}" failed. No retries permitted until 2026-09-06 12:47:37.601006297 +0000 UTC m=+67.343377444 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs") pod "router-default-659dc76f98-xl7g6" (UID: "75be41fd-be89-4ca5-9bab-ac64e74295ba") : secret "router-metrics-certs-default" not found Sep 06 12:47:21.662992 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:21.662960 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-s6ght_ab6850db-dfc3-47f9-b91b-54757857d601/kube-storage-version-migrator-operator/0.log" Sep 06 12:47:25.236831 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:25.236792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:25.236831 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:25.236834 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:25.237356 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:25.236927 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 06 12:47:25.237356 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:25.236988 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls podName:5d654b09-beec-407d-b4b7-cc802e221d43 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:33.236975565 +0000 UTC m=+62.979346697 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-462kn" (UID: "5d654b09-beec-407d-b4b7-cc802e221d43") : secret "insights-runtime-extractor-tls" not found Sep 06 12:47:25.237356 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:25.237320 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d654b09-beec-407d-b4b7-cc802e221d43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:26.025609 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:26.025556 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:26.025789 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:26.025633 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:26.025997 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:26.025984 2580 scope.go:117] "RemoveContainer" containerID="512543a62434237f32385bc61fbaf544f11382fda1c4700a39d440e7a28b749d" Sep 06 12:47:26.026152 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:26.026134 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-2vfhs_openshift-console-operator(8750b827-635a-4805-817c-45a970d9fc81)\"" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" podUID="8750b827-635a-4805-817c-45a970d9fc81" Sep 06 12:47:28.365766 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:28.365727 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d2c2j\" (UID: \"c28de569-a5a3-46dc-b764-7244ecb5b380\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:28.366243 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:28.365901 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 06 12:47:28.366243 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:47:28.365988 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert podName:c28de569-a5a3-46dc-b764-7244ecb5b380 nodeName:}" failed. No retries permitted until 2026-09-06 12:47:44.365967842 +0000 UTC m=+74.108338976 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-d2c2j" (UID: "c28de569-a5a3-46dc-b764-7244ecb5b380") : secret "networking-console-plugin-cert" not found Sep 06 12:47:30.972470 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:30.972438 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4p8dj" Sep 06 12:47:33.310235 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:33.310196 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:33.312519 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:33.312492 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d654b09-beec-407d-b4b7-cc802e221d43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-462kn\" (UID: \"5d654b09-beec-407d-b4b7-cc802e221d43\") " pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:33.439369 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:33.439341 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mb5b8\"" Sep 06 12:47:33.447938 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:33.447921 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-462kn" Sep 06 12:47:33.717103 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:33.717079 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-462kn"] Sep 06 12:47:33.719352 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:33.719323 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d654b09_beec_407d_b4b7_cc802e221d43.slice/crio-5f793490384921ac0b345fe72c27bc92e16ece7fd0c22af6c4839b85ee0f20ed WatchSource:0}: Error finding container 5f793490384921ac0b345fe72c27bc92e16ece7fd0c22af6c4839b85ee0f20ed: Status 404 returned error can't find the container with id 5f793490384921ac0b345fe72c27bc92e16ece7fd0c22af6c4839b85ee0f20ed Sep 06 12:47:34.134378 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:34.134342 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-462kn" event={"ID":"5d654b09-beec-407d-b4b7-cc802e221d43","Type":"ContainerStarted","Data":"58fda434f75472881a5c8c30253e358195ed73e9aa33dbd4acbac040263c78df"} Sep 06 12:47:34.134543 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:34.134398 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-462kn" event={"ID":"5d654b09-beec-407d-b4b7-cc802e221d43","Type":"ContainerStarted","Data":"5f793490384921ac0b345fe72c27bc92e16ece7fd0c22af6c4839b85ee0f20ed"} Sep 06 12:47:35.140441 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:35.140400 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-462kn" event={"ID":"5d654b09-beec-407d-b4b7-cc802e221d43","Type":"ContainerStarted","Data":"245175276b8dd4eae7803dea1790f536684cedcf471dea6f182391d5acf354ae"} Sep 06 12:47:36.146128 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.146095 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-462kn" event={"ID":"5d654b09-beec-407d-b4b7-cc802e221d43","Type":"ContainerStarted","Data":"e1bfdc7dc038b7da88419c157f33bb2059d12080fc1fd770fed31cf1f0c37f1e"} Sep 06 12:47:36.162890 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.162828 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-462kn" podStartSLOduration=17.038921684 podStartE2EDuration="19.162814016s" totalImagesPullingTime="2.123892332s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 12:47:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:36.162164592 +0000 UTC m=+65.904535748" watchObservedRunningTime="2026-09-06 12:47:36.162814016 +0000 UTC m=+65.905185171" Sep 06 12:47:36.440486 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.440448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:47:36.442703 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.442685 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 06 12:47:36.453513 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.453485 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d02b82-c6e5-4400-b240-dd933ab712a1-metrics-certs\") pod \"network-metrics-daemon-5t886\" (UID: \"42d02b82-c6e5-4400-b240-dd933ab712a1\") " pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:47:36.541036 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.540982 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:47:36.541198 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.541129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm4q2\" (UniqueName: \"kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2\") pod \"network-check-target-x89d8\" (UID: \"0027ef15-df27-4284-9cec-5fa0b603abf1\") " pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:47:36.543658 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.543631 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 06 12:47:36.543765 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.543667 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm4q2\" (UniqueName: \"kubernetes.io/projected/0027ef15-df27-4284-9cec-5fa0b603abf1-kube-api-access-cm4q2\") pod \"network-check-target-x89d8\" (UID: \"0027ef15-df27-4284-9cec-5fa0b603abf1\") " pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:47:36.553461 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.553435 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5823a60b-e18f-4f3c-b8f5-07cf5ce58135-original-pull-secret\") pod \"global-pull-secret-syncer-d9f5r\" (UID: \"5823a60b-e18f-4f3c-b8f5-07cf5ce58135\") " pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:47:36.652193 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.652161 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ghsp2\"" Sep 06 12:47:36.659743 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.659721 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nqwjz\"" Sep 06 12:47:36.660763 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.660747 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5t886" Sep 06 12:47:36.664564 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.664550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9f5r" Sep 06 12:47:36.668333 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.668312 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:47:36.949214 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.948109 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5t886"] Sep 06 12:47:36.951729 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:36.951691 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d02b82_c6e5_4400_b240_dd933ab712a1.slice/crio-e5fc4b1976cb0e81272b76f2e1e5b4a0d95f230f4366f0f9bce8b6bb1b62c97e WatchSource:0}: Error finding container e5fc4b1976cb0e81272b76f2e1e5b4a0d95f230f4366f0f9bce8b6bb1b62c97e: Status 404 returned error can't find the container with id e5fc4b1976cb0e81272b76f2e1e5b4a0d95f230f4366f0f9bce8b6bb1b62c97e Sep 06 12:47:36.953315 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.953293 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d9f5r"] Sep 06 12:47:36.956111 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:36.956090 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5823a60b_e18f_4f3c_b8f5_07cf5ce58135.slice/crio-aab3249fe03897560b8ddd24cc7c08022e6a35517fea69a47cff157729177921 WatchSource:0}: Error finding container aab3249fe03897560b8ddd24cc7c08022e6a35517fea69a47cff157729177921: Status 404 returned error can't find the container with id aab3249fe03897560b8ddd24cc7c08022e6a35517fea69a47cff157729177921 Sep 06 12:47:36.967780 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:36.967758 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x89d8"] Sep 06 12:47:36.970595 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:36.970567 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0027ef15_df27_4284_9cec_5fa0b603abf1.slice/crio-c6b1f1d95e2505afbcc1ddcba813d9104b98b3e3ba6004e8d2979cacc77137b2 WatchSource:0}: Error finding container c6b1f1d95e2505afbcc1ddcba813d9104b98b3e3ba6004e8d2979cacc77137b2: Status 404 returned error can't find the container with id c6b1f1d95e2505afbcc1ddcba813d9104b98b3e3ba6004e8d2979cacc77137b2 Sep 06 12:47:37.150834 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.150801 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d9f5r" event={"ID":"5823a60b-e18f-4f3c-b8f5-07cf5ce58135","Type":"ContainerStarted","Data":"aab3249fe03897560b8ddd24cc7c08022e6a35517fea69a47cff157729177921"} Sep 06 12:47:37.151921 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.151893 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5t886" event={"ID":"42d02b82-c6e5-4400-b240-dd933ab712a1","Type":"ContainerStarted","Data":"e5fc4b1976cb0e81272b76f2e1e5b4a0d95f230f4366f0f9bce8b6bb1b62c97e"} Sep 06 12:47:37.153390 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.153368 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x89d8" event={"ID":"0027ef15-df27-4284-9cec-5fa0b603abf1","Type":"ContainerStarted","Data":"7e3f6e67ec7c5cbca2c1d5ad533cd8270578ef07ef514ce3bb60ce08b604f455"} Sep 06 12:47:37.153476 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.153406 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x89d8" event={"ID":"0027ef15-df27-4284-9cec-5fa0b603abf1","Type":"ContainerStarted","Data":"c6b1f1d95e2505afbcc1ddcba813d9104b98b3e3ba6004e8d2979cacc77137b2"} Sep 06 12:47:37.169910 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.169867 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-x89d8" podStartSLOduration=66.169853832 podStartE2EDuration="1m6.169853832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:37.169530117 +0000 UTC m=+66.911901275" watchObservedRunningTime="2026-09-06 12:47:37.169853832 +0000 UTC m=+66.912224994" Sep 06 12:47:37.650553 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.650512 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:37.650737 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.650596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:37.650737 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.650622 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:37.650737 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.650650 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:37.650737 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.650681 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:37.650737 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.650708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-b5zlx\" (UID: \"0c15574a-7aa1-4f24-b884-90ec43811ef2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:37.653772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.651897 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75be41fd-be89-4ca5-9bab-ac64e74295ba-service-ca-bundle\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:37.653772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.653612 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c15574a-7aa1-4f24-b884-90ec43811ef2-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-b5zlx\" (UID: \"0c15574a-7aa1-4f24-b884-90ec43811ef2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:37.653772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.653682 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls\") pod \"image-registry-6777498f4c-ccqwz\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:37.653772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.653684 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75be41fd-be89-4ca5-9bab-ac64e74295ba-metrics-certs\") pod \"router-default-659dc76f98-xl7g6\" (UID: \"75be41fd-be89-4ca5-9bab-ac64e74295ba\") " pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:37.653956 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.653911 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c34f36a1-7b51-43f1-a481-e5331b2ee206-metrics-tls\") pod \"dns-default-fgcpf\" (UID: \"c34f36a1-7b51-43f1-a481-e5331b2ee206\") " pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:37.654606 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.654580 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8dd16fa-c99c-4c68-b472-4927f82d5e5a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9gph2\" (UID: \"b8dd16fa-c99c-4c68-b472-4927f82d5e5a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:37.772400 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.772364 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tnxck\"" Sep 06 12:47:37.779986 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.779953 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:37.794284 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.794175 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sk4dn\"" Sep 06 12:47:37.802168 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.802136 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" Sep 06 12:47:37.817633 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.817604 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nzkhb\"" Sep 06 12:47:37.825535 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.825471 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" Sep 06 12:47:37.839606 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.839330 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-st85w\"" Sep 06 12:47:37.850739 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.850698 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:37.875597 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.875377 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g7xpz\"" Sep 06 12:47:37.886679 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:37.886329 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:38.058099 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.057889 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6777498f4c-ccqwz"] Sep 06 12:47:38.061535 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:38.061495 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf220f886_f2be_4f33_b8fc_1d0f3dd3dafd.slice/crio-985af56ef8577c1d0ff9d5d5a49f31d5947633dc96e1526adb9d92657c207505 WatchSource:0}: Error finding container 985af56ef8577c1d0ff9d5d5a49f31d5947633dc96e1526adb9d92657c207505: Status 404 returned error can't find the container with id 985af56ef8577c1d0ff9d5d5a49f31d5947633dc96e1526adb9d92657c207505 Sep 06 12:47:38.093805 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.093711 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx"] Sep 06 12:47:38.129630 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.129603 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2"] Sep 06 12:47:38.134195 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:38.134168 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8dd16fa_c99c_4c68_b472_4927f82d5e5a.slice/crio-32cd2397549f75e3c9d8d31365c89e3ae39ff5c43c5df06801fb8e2d37e10d13 WatchSource:0}: Error finding container 32cd2397549f75e3c9d8d31365c89e3ae39ff5c43c5df06801fb8e2d37e10d13: Status 404 returned error can't find the container with id 32cd2397549f75e3c9d8d31365c89e3ae39ff5c43c5df06801fb8e2d37e10d13 Sep 06 12:47:38.156389 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.156124 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-659dc76f98-xl7g6"] Sep 06 12:47:38.163331 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:38.161940 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75be41fd_be89_4ca5_9bab_ac64e74295ba.slice/crio-417d1c090263567f513c24ff5310e5b31204e6389ad000042468170ec4dfa173 WatchSource:0}: Error finding container 417d1c090263567f513c24ff5310e5b31204e6389ad000042468170ec4dfa173: Status 404 returned error can't find the container with id 417d1c090263567f513c24ff5310e5b31204e6389ad000042468170ec4dfa173 Sep 06 12:47:38.166653 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.164395 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" event={"ID":"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd","Type":"ContainerStarted","Data":"e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668"} Sep 06 12:47:38.166653 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.164456 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" event={"ID":"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd","Type":"ContainerStarted","Data":"985af56ef8577c1d0ff9d5d5a49f31d5947633dc96e1526adb9d92657c207505"} Sep 06 12:47:38.166653 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.164916 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:47:38.167224 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.167092 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" event={"ID":"0c15574a-7aa1-4f24-b884-90ec43811ef2","Type":"ContainerStarted","Data":"302e8b209445984cd58ee21a153e81c0c7c67811c16a5de45fc85d9b0c0140a1"} Sep 06 12:47:38.169307 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.168992 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" event={"ID":"b8dd16fa-c99c-4c68-b472-4927f82d5e5a","Type":"ContainerStarted","Data":"32cd2397549f75e3c9d8d31365c89e3ae39ff5c43c5df06801fb8e2d37e10d13"} Sep 06 12:47:38.170288 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.169757 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:47:38.184805 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.184543 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fgcpf"] Sep 06 12:47:38.186458 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.186320 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" podStartSLOduration=67.186299628 podStartE2EDuration="1m7.186299628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:38.186184094 +0000 UTC m=+67.928555252" watchObservedRunningTime="2026-09-06 12:47:38.186299628 +0000 UTC m=+67.928670784" Sep 06 12:47:38.190229 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:38.189697 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc34f36a1_7b51_43f1_a481_e5331b2ee206.slice/crio-1cb53eada6b3ffc6e3650d319eca2ceccd6b04ac1f9de8dce13adfbb879fe7e7 WatchSource:0}: Error finding container 1cb53eada6b3ffc6e3650d319eca2ceccd6b04ac1f9de8dce13adfbb879fe7e7: Status 404 returned error can't find the container with id 1cb53eada6b3ffc6e3650d319eca2ceccd6b04ac1f9de8dce13adfbb879fe7e7 Sep 06 12:47:38.613863 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.613804 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v"] Sep 06 12:47:38.626534 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.626456 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz"] Sep 06 12:47:38.626708 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.626613 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:38.631572 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.631545 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 06 12:47:38.631885 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.631863 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 06 12:47:38.633299 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.632411 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 06 12:47:38.633299 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.632642 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 06 12:47:38.636551 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.636528 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds"] Sep 06 12:47:38.636840 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.636791 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz" Sep 06 12:47:38.639089 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.639067 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bmzrg\"" Sep 06 12:47:38.639432 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.639412 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 06 12:47:38.651847 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.645873 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz"] Sep 06 12:47:38.651847 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.645937 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v"] Sep 06 12:47:38.651847 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.645970 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds"] Sep 06 12:47:38.651847 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.646128 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.654914 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.654631 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 06 12:47:38.657692 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.655239 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 06 12:47:38.657692 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.655804 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 06 12:47:38.663093 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.663067 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 06 12:47:38.730399 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.730367 2580 scope.go:117] "RemoveContainer" containerID="512543a62434237f32385bc61fbaf544f11382fda1c4700a39d440e7a28b749d" Sep 06 12:47:38.763305 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.763244 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f4ff0ca-681d-4c0d-8036-e04bf222faae-additional-service-ca\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.763491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.763332 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d-tmp\") pod \"klusterlet-addon-workmgr-77998c6665-xvc6v\" (UID: \"a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:38.763491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.763358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3f4ff0ca-681d-4c0d-8036-e04bf222faae-hub\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.763491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.763393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj5df\" (UniqueName: \"kubernetes.io/projected/a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d-kube-api-access-nj5df\") pod \"klusterlet-addon-workmgr-77998c6665-xvc6v\" (UID: \"a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:38.763491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.763447 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xjcq\" (UniqueName: \"kubernetes.io/projected/3f4ff0ca-681d-4c0d-8036-e04bf222faae-kube-api-access-9xjcq\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.763491 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.763476 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f4ff0ca-681d-4c0d-8036-e04bf222faae-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.763700 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.763514 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bljb\" (UniqueName: \"kubernetes.io/projected/e7954306-4aa7-4d73-9bf7-094552c588a6-kube-api-access-9bljb\") pod \"managed-serviceaccount-addon-agent-59b97db99d-7qjdz\" (UID: \"e7954306-4aa7-4d73-9bf7-094552c588a6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz" Sep 06 12:47:38.763700 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.763611 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7954306-4aa7-4d73-9bf7-094552c588a6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59b97db99d-7qjdz\" (UID: \"e7954306-4aa7-4d73-9bf7-094552c588a6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz" Sep 06 12:47:38.763700 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.763647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3f4ff0ca-681d-4c0d-8036-e04bf222faae-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.763700 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.763687 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d-klusterlet-config\") pod \"klusterlet-addon-workmgr-77998c6665-xvc6v\" (UID: \"a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:38.763856 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.763717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3f4ff0ca-681d-4c0d-8036-e04bf222faae-ca\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.864369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f4ff0ca-681d-4c0d-8036-e04bf222faae-additional-service-ca\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.864405 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d-tmp\") pod \"klusterlet-addon-workmgr-77998c6665-xvc6v\" (UID: \"a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.864422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3f4ff0ca-681d-4c0d-8036-e04bf222faae-hub\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.864566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj5df\" (UniqueName: \"kubernetes.io/projected/a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d-kube-api-access-nj5df\") pod \"klusterlet-addon-workmgr-77998c6665-xvc6v\" (UID: \"a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.864660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xjcq\" (UniqueName: \"kubernetes.io/projected/3f4ff0ca-681d-4c0d-8036-e04bf222faae-kube-api-access-9xjcq\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.864694 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f4ff0ca-681d-4c0d-8036-e04bf222faae-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.864751 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bljb\" (UniqueName: \"kubernetes.io/projected/e7954306-4aa7-4d73-9bf7-094552c588a6-kube-api-access-9bljb\") pod \"managed-serviceaccount-addon-agent-59b97db99d-7qjdz\" (UID: \"e7954306-4aa7-4d73-9bf7-094552c588a6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.864784 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d-tmp\") pod \"klusterlet-addon-workmgr-77998c6665-xvc6v\" (UID: \"a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.864815 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7954306-4aa7-4d73-9bf7-094552c588a6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59b97db99d-7qjdz\" (UID: \"e7954306-4aa7-4d73-9bf7-094552c588a6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.864839 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3f4ff0ca-681d-4c0d-8036-e04bf222faae-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.864882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d-klusterlet-config\") pod \"klusterlet-addon-workmgr-77998c6665-xvc6v\" (UID: \"a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.864904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3f4ff0ca-681d-4c0d-8036-e04bf222faae-ca\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.865225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.865074 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f4ff0ca-681d-4c0d-8036-e04bf222faae-additional-service-ca\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.867674 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.867623 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3f4ff0ca-681d-4c0d-8036-e04bf222faae-hub\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.867900 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.867881 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7954306-4aa7-4d73-9bf7-094552c588a6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59b97db99d-7qjdz\" (UID: \"e7954306-4aa7-4d73-9bf7-094552c588a6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz" Sep 06 12:47:38.868658 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.868622 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3f4ff0ca-681d-4c0d-8036-e04bf222faae-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.870221 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.869168 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3f4ff0ca-681d-4c0d-8036-e04bf222faae-ca\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.870554 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.870382 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d-klusterlet-config\") pod \"klusterlet-addon-workmgr-77998c6665-xvc6v\" (UID: \"a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:38.871180 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.870967 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f4ff0ca-681d-4c0d-8036-e04bf222faae-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.885400 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.885369 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bljb\" (UniqueName: \"kubernetes.io/projected/e7954306-4aa7-4d73-9bf7-094552c588a6-kube-api-access-9bljb\") pod \"managed-serviceaccount-addon-agent-59b97db99d-7qjdz\" (UID: \"e7954306-4aa7-4d73-9bf7-094552c588a6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz" Sep 06 12:47:38.885538 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.885519 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xjcq\" (UniqueName: \"kubernetes.io/projected/3f4ff0ca-681d-4c0d-8036-e04bf222faae-kube-api-access-9xjcq\") pod \"cluster-proxy-proxy-agent-788954c877-r9sds\" (UID: \"3f4ff0ca-681d-4c0d-8036-e04bf222faae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:38.885696 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.885676 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj5df\" (UniqueName: \"kubernetes.io/projected/a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d-kube-api-access-nj5df\") pod \"klusterlet-addon-workmgr-77998c6665-xvc6v\" (UID: \"a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:38.946354 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.946225 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:38.967677 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.967493 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz" Sep 06 12:47:38.975189 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:38.975097 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" Sep 06 12:47:39.180037 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.179369 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5t886" event={"ID":"42d02b82-c6e5-4400-b240-dd933ab712a1","Type":"ContainerStarted","Data":"0ce78dcfe993b909740085ea64520b4f0ad7937ebde5ce575f8f7c2ca860f972"} Sep 06 12:47:39.180037 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.179436 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5t886" event={"ID":"42d02b82-c6e5-4400-b240-dd933ab712a1","Type":"ContainerStarted","Data":"31d4a25be33571132e9147b422da0baccb987c0b12be5c5e4e6394ceb5b132a4"} Sep 06 12:47:39.182040 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.181964 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fgcpf" event={"ID":"c34f36a1-7b51-43f1-a481-e5331b2ee206","Type":"ContainerStarted","Data":"1cb53eada6b3ffc6e3650d319eca2ceccd6b04ac1f9de8dce13adfbb879fe7e7"} Sep 06 12:47:39.184856 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.184684 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 12:47:39.184856 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.184777 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" event={"ID":"8750b827-635a-4805-817c-45a970d9fc81","Type":"ContainerStarted","Data":"bb2530307a530e3504ab7aeddeb6a69ac7c695bf0b2864528b72fa42d966f8ce"} Sep 06 12:47:39.186863 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.185848 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:39.188231 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.187577 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-659dc76f98-xl7g6" event={"ID":"75be41fd-be89-4ca5-9bab-ac64e74295ba","Type":"ContainerStarted","Data":"48cc22a155e2e56c879999471a1c597f79d9ab2b08ba491a3a1f04a253ff2340"} Sep 06 12:47:39.188231 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.187625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-659dc76f98-xl7g6" event={"ID":"75be41fd-be89-4ca5-9bab-ac64e74295ba","Type":"ContainerStarted","Data":"417d1c090263567f513c24ff5310e5b31204e6389ad000042468170ec4dfa173"} Sep 06 12:47:39.203710 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.202769 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5t886" podStartSLOduration=66.372881884 podStartE2EDuration="1m8.202749439s" totalImagesPullingTime="1.829867555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:39.202367249 +0000 UTC m=+68.944738405" watchObservedRunningTime="2026-09-06 12:47:39.202749439 +0000 UTC m=+68.945120595" Sep 06 12:47:39.233717 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.233164 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" podStartSLOduration=53.574964427 podStartE2EDuration="1m1.233146067s" totalImagesPullingTime="7.65818164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:39.232148415 +0000 UTC m=+68.974519570" watchObservedRunningTime="2026-09-06 12:47:39.233146067 +0000 UTC m=+68.975517224" Sep 06 12:47:39.259207 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.258363 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-659dc76f98-xl7g6" podStartSLOduration=62.25834503 podStartE2EDuration="1m2.25834503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:39.256167909 +0000 UTC m=+68.998539065" watchObservedRunningTime="2026-09-06 12:47:39.25834503 +0000 UTC m=+69.000716181" Sep 06 12:47:39.284410 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.284362 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v"] Sep 06 12:47:39.288290 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:39.288178 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda85a54f4_84f1_45a4_b6b8_a0abf5e6cc6d.slice/crio-8b53cd0001afe08d1082c8136218d898e80c0feddb540a400d8ec56f9637c83f WatchSource:0}: Error finding container 8b53cd0001afe08d1082c8136218d898e80c0feddb540a400d8ec56f9637c83f: Status 404 returned error can't find the container with id 8b53cd0001afe08d1082c8136218d898e80c0feddb540a400d8ec56f9637c83f Sep 06 12:47:39.335450 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.333503 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz"] Sep 06 12:47:39.339026 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:39.338993 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7954306_4aa7_4d73_9bf7_094552c588a6.slice/crio-461204ea7ec39698d48ccceafb15e26eb6e35798bd749cb138e812bdb3de6c7d WatchSource:0}: Error finding container 461204ea7ec39698d48ccceafb15e26eb6e35798bd749cb138e812bdb3de6c7d: Status 404 returned error can't find the container with id 461204ea7ec39698d48ccceafb15e26eb6e35798bd749cb138e812bdb3de6c7d Sep 06 12:47:39.364252 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.364214 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds"] Sep 06 12:47:39.373394 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:47:39.373360 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f4ff0ca_681d_4c0d_8036_e04bf222faae.slice/crio-7752e3c73a1230c659a3e9e8d5a0eb4bec265f11cb684d77e273d84cb02eedff WatchSource:0}: Error finding container 7752e3c73a1230c659a3e9e8d5a0eb4bec265f11cb684d77e273d84cb02eedff: Status 404 returned error can't find the container with id 7752e3c73a1230c659a3e9e8d5a0eb4bec265f11cb684d77e273d84cb02eedff Sep 06 12:47:39.617362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.617213 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-2vfhs" Sep 06 12:47:39.853633 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.852132 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:39.855670 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:39.855641 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:40.197385 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:40.197280 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" event={"ID":"3f4ff0ca-681d-4c0d-8036-e04bf222faae","Type":"ContainerStarted","Data":"7752e3c73a1230c659a3e9e8d5a0eb4bec265f11cb684d77e273d84cb02eedff"} Sep 06 12:47:40.200944 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:40.200909 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz" event={"ID":"e7954306-4aa7-4d73-9bf7-094552c588a6","Type":"ContainerStarted","Data":"461204ea7ec39698d48ccceafb15e26eb6e35798bd749cb138e812bdb3de6c7d"} Sep 06 12:47:40.209040 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:40.207482 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" event={"ID":"a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d","Type":"ContainerStarted","Data":"8b53cd0001afe08d1082c8136218d898e80c0feddb540a400d8ec56f9637c83f"} Sep 06 12:47:40.209321 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:40.208980 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:40.211131 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:40.210890 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-659dc76f98-xl7g6" Sep 06 12:47:44.428537 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:44.428493 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d2c2j\" (UID: \"c28de569-a5a3-46dc-b764-7244ecb5b380\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:44.431415 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:44.431382 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c28de569-a5a3-46dc-b764-7244ecb5b380-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d2c2j\" (UID: \"c28de569-a5a3-46dc-b764-7244ecb5b380\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:44.627136 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:44.627102 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xjzzn\"" Sep 06 12:47:44.636026 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:44.635980 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" Sep 06 12:47:49.592058 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:49.592015 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-d2c2j"] Sep 06 12:47:50.259763 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.259729 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" event={"ID":"a85a54f4-84f1-45a4-b6b8-a0abf5e6cc6d","Type":"ContainerStarted","Data":"da813718dda32b81274b1706b6dd6e990ad038c82a239c9cba5a8db8911318db"} Sep 06 12:47:50.260131 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.260105 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:50.262119 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.262070 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" Sep 06 12:47:50.262895 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.262809 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" event={"ID":"3f4ff0ca-681d-4c0d-8036-e04bf222faae","Type":"ContainerStarted","Data":"21b6a95610bd710ad2e51fb251337ca039b39300ab4fc36088440a145efe1958"} Sep 06 12:47:50.262895 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.262852 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" event={"ID":"3f4ff0ca-681d-4c0d-8036-e04bf222faae","Type":"ContainerStarted","Data":"99a87bba82450a2baf99ad6964c8b28c00290502eb9f1c046102d4b354d77804"} Sep 06 12:47:50.262895 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.262872 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" event={"ID":"3f4ff0ca-681d-4c0d-8036-e04bf222faae","Type":"ContainerStarted","Data":"37ebd453c960d07728c3e05ffe518d64ec55e388840206b2ca7ef4518f128ede"} Sep 06 12:47:50.265001 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.264977 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d9f5r" event={"ID":"5823a60b-e18f-4f3c-b8f5-07cf5ce58135","Type":"ContainerStarted","Data":"dd1a827be2dce799794e5a73f78a039ed148ecb3de3e11c3976d82b6209bc124"} Sep 06 12:47:50.267054 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.267029 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" event={"ID":"0c15574a-7aa1-4f24-b884-90ec43811ef2","Type":"ContainerStarted","Data":"b504fd23fcd912dbed505fa65fccfd6aee28634066fde0ee189f187c49d2c7c2"} Sep 06 12:47:50.267140 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.267080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" event={"ID":"0c15574a-7aa1-4f24-b884-90ec43811ef2","Type":"ContainerStarted","Data":"daddad25c9cf841a0ca58ed70cc3e910646b67ca9b69476850a15ab2be55a79d"} Sep 06 12:47:50.268812 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.268779 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" event={"ID":"b8dd16fa-c99c-4c68-b472-4927f82d5e5a","Type":"ContainerStarted","Data":"548ad26ec42d4bd261d5a32c5e2b777bfd248b7cbeeeb2883d43c57d83291148"} Sep 06 12:47:50.270768 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.270743 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fgcpf" event={"ID":"c34f36a1-7b51-43f1-a481-e5331b2ee206","Type":"ContainerStarted","Data":"d9ee15c0baa3d55bb1aea78e213caa21d0081aa6f62ff2a32a38c0dc4b46352f"} Sep 06 12:47:50.270863 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.270786 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fgcpf" event={"ID":"c34f36a1-7b51-43f1-a481-e5331b2ee206","Type":"ContainerStarted","Data":"e93260dc280f2418eab3be236df19ff31dcfbef2f57e07e3d33c1595b5d4e5c2"} Sep 06 12:47:50.270863 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.270821 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fgcpf" Sep 06 12:47:50.272161 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.272137 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" event={"ID":"c28de569-a5a3-46dc-b764-7244ecb5b380","Type":"ContainerStarted","Data":"3b016f636c223b9cba59854e41be3cdd853d90d2dc47f77f200010bf303e7294"} Sep 06 12:47:50.273764 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.273734 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz" event={"ID":"e7954306-4aa7-4d73-9bf7-094552c588a6","Type":"ContainerStarted","Data":"dd418493de9afee95dd0a99c82f634074373fda785eeb94718c3daef7576b0bd"} Sep 06 12:47:50.276566 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.276531 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77998c6665-xvc6v" podStartSLOduration=2.200724874 podStartE2EDuration="12.276518255s" totalImagesPullingTime="10.075793381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:50.274846336 +0000 UTC m=+80.017217501" watchObservedRunningTime="2026-09-06 12:47:50.276518255 +0000 UTC m=+80.018889409" Sep 06 12:47:50.292062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.292007 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fgcpf" podStartSLOduration=34.177438866 podStartE2EDuration="45.291989027s" totalImagesPullingTime="11.114550161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:47:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:50.290105369 +0000 UTC m=+80.032476525" watchObservedRunningTime="2026-09-06 12:47:50.291989027 +0000 UTC m=+80.034360183" Sep 06 12:47:50.307092 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.307038 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-d9f5r" podStartSLOduration=66.954118552 podStartE2EDuration="1m19.307022992s" totalImagesPullingTime="12.35290444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 12:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:50.306512549 +0000 UTC m=+80.048883705" watchObservedRunningTime="2026-09-06 12:47:50.307022992 +0000 UTC m=+80.049394139" Sep 06 12:47:50.327140 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.327088 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9gph2" podStartSLOduration=62.153220065 podStartE2EDuration="1m13.327070231s" totalImagesPullingTime="11.173850166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:50.32592258 +0000 UTC m=+80.068293735" watchObservedRunningTime="2026-09-06 12:47:50.327070231 +0000 UTC m=+80.069441386" Sep 06 12:47:50.343321 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.343277 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-b5zlx" podStartSLOduration=62.184983874 podStartE2EDuration="1m13.343248999s" totalImagesPullingTime="11.158265125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:50.342715773 +0000 UTC m=+80.085086927" watchObservedRunningTime="2026-09-06 12:47:50.343248999 +0000 UTC m=+80.085620153" Sep 06 12:47:50.364626 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.364578 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-788954c877-r9sds" podStartSLOduration=2.4042012870000002 podStartE2EDuration="12.364562715s" totalImagesPullingTime="9.960361428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:50.362977832 +0000 UTC m=+80.105348988" watchObservedRunningTime="2026-09-06 12:47:50.364562715 +0000 UTC m=+80.106933871" Sep 06 12:47:50.378144 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:50.378106 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59b97db99d-7qjdz" podStartSLOduration=2.384291314 podStartE2EDuration="12.378091019s" totalImagesPullingTime="9.993799705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:50.377189587 +0000 UTC m=+80.119560741" watchObservedRunningTime="2026-09-06 12:47:50.378091019 +0000 UTC m=+80.120462175" Sep 06 12:47:51.279433 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:51.279379 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" event={"ID":"c28de569-a5a3-46dc-b764-7244ecb5b380","Type":"ContainerStarted","Data":"dd471837c1e2168679339988cfd7a9513e139e8a72c7c704145622eeb2aa9164"} Sep 06 12:47:51.296638 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:51.296592 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-d2c2j" podStartSLOduration=38.15426482 podStartE2EDuration="39.296577112s" totalImagesPullingTime="1.142312292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:47:51.295611901 +0000 UTC m=+81.037983057" watchObservedRunningTime="2026-09-06 12:47:51.296577112 +0000 UTC m=+81.038948320" Sep 06 12:47:57.785019 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:57.784893 2580 patch_prober.go:28] interesting pod/image-registry-6777498f4c-ccqwz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 06 12:47:57.785019 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:47:57.784963 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" podUID="f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 06 12:48:00.212477 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:00.212448 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:48:00.282983 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:00.282954 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fgcpf" Sep 06 12:48:01.042167 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:01.042130 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6777498f4c-ccqwz"] Sep 06 12:48:05.435053 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.435020 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pwz9w"] Sep 06 12:48:05.438068 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.438046 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.440475 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.440451 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 06 12:48:05.441487 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.440978 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vdm9k\"" Sep 06 12:48:05.441790 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.441620 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 06 12:48:05.441790 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.441683 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 06 12:48:05.442027 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.441934 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 06 12:48:05.499352 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.499322 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-wtmp\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.499564 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.499536 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.499645 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.499579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-tls\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.499645 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.499637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-sys\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.499743 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.499713 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-root\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.499804 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.499747 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.499804 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.499797 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-metrics-client-ca\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.499903 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.499826 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q2zj\" (UniqueName: \"kubernetes.io/projected/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-kube-api-access-2q2zj\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.499903 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.499858 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-textfile\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.600525 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.600481 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-root\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.600525 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.600531 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.600772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.600566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-metrics-client-ca\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.600772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.600591 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q2zj\" (UniqueName: \"kubernetes.io/projected/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-kube-api-access-2q2zj\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.600772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.600597 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-root\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.600772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.600621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-textfile\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.600772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.600691 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-wtmp\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.600772 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.600713 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.601055 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.600861 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-wtmp\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.601055 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.600914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-tls\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.601055 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.600982 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-sys\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.601055 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.601053 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-sys\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.601251 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.601079 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-textfile\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.601251 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.601240 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-metrics-client-ca\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.601368 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.601353 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.603015 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.602996 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.603245 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.603228 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-node-exporter-tls\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.608592 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.608571 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q2zj\" (UniqueName: \"kubernetes.io/projected/d2b2e3af-d7c9-49f0-a5d0-bfc985417b52-kube-api-access-2q2zj\") pod \"node-exporter-pwz9w\" (UID: \"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52\") " pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.749635 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:05.749564 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pwz9w" Sep 06 12:48:05.761666 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:48:05.761628 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b2e3af_d7c9_49f0_a5d0_bfc985417b52.slice/crio-75c4f78b2f661a2926042068e98c6c470a79b9ec7fd14c4dbc11505410858a35 WatchSource:0}: Error finding container 75c4f78b2f661a2926042068e98c6c470a79b9ec7fd14c4dbc11505410858a35: Status 404 returned error can't find the container with id 75c4f78b2f661a2926042068e98c6c470a79b9ec7fd14c4dbc11505410858a35 Sep 06 12:48:06.340837 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:06.340790 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwz9w" event={"ID":"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52","Type":"ContainerStarted","Data":"75c4f78b2f661a2926042068e98c6c470a79b9ec7fd14c4dbc11505410858a35"} Sep 06 12:48:07.346381 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:07.346344 2580 generic.go:356] "Generic (PLEG): container finished" podID="d2b2e3af-d7c9-49f0-a5d0-bfc985417b52" containerID="702ad126fdba18754d811f628f8c111bc0fa5c3101a85b8851a1bb8bf89e4bea" exitCode=0 Sep 06 12:48:07.346736 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:07.346397 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwz9w" event={"ID":"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52","Type":"ContainerDied","Data":"702ad126fdba18754d811f628f8c111bc0fa5c3101a85b8851a1bb8bf89e4bea"} Sep 06 12:48:08.353127 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:08.353085 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwz9w" event={"ID":"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52","Type":"ContainerStarted","Data":"3d823cb7fa0da9a40fd9e6a3d693380c5e2b6d0d9855701f80a66d37a59c4d8d"} Sep 06 12:48:08.353553 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:08.353138 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwz9w" event={"ID":"d2b2e3af-d7c9-49f0-a5d0-bfc985417b52","Type":"ContainerStarted","Data":"84ee069d824dc1ed1f118e427311f8badaa7ca4c06622660294f783776f012b4"} Sep 06 12:48:08.376035 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:08.375980 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pwz9w" podStartSLOduration=2.275098367 podStartE2EDuration="3.375964158s" totalImagesPullingTime="1.03730511s" totalInitContainerRuntime="63.560681ms" isStatefulPod=true podCreationTimestamp="2026-09-06 12:48:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:48:08.374589491 +0000 UTC m=+98.116960647" watchObservedRunningTime="2026-09-06 12:48:08.375964158 +0000 UTC m=+98.118335312" Sep 06 12:48:09.190818 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:09.190788 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-x89d8" Sep 06 12:48:26.062322 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.062280 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" podUID="f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" containerName="registry" containerID="cri-o://e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668" gracePeriod=30 Sep 06 12:48:26.316538 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.316481 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:48:26.424336 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.424302 2580 generic.go:356] "Generic (PLEG): container finished" podID="f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" containerID="e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668" exitCode=0 Sep 06 12:48:26.424487 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.424347 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" Sep 06 12:48:26.424487 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.424355 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" event={"ID":"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd","Type":"ContainerDied","Data":"e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668"} Sep 06 12:48:26.424487 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.424404 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6777498f4c-ccqwz" event={"ID":"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd","Type":"ContainerDied","Data":"985af56ef8577c1d0ff9d5d5a49f31d5947633dc96e1526adb9d92657c207505"} Sep 06 12:48:26.424487 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.424430 2580 scope.go:117] "RemoveContainer" containerID="e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668" Sep 06 12:48:26.432701 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.432682 2580 scope.go:117] "RemoveContainer" containerID="e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668" Sep 06 12:48:26.432990 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:48:26.432972 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668\": container with ID starting with e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668 not found: ID does not exist" containerID="e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668" Sep 06 12:48:26.433033 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.432997 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668"} err="failed to get container status \"e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668\": rpc error: code = NotFound desc = could not find container \"e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668\": container with ID starting with e1dd3b4021c3ffb3ed55f4546b09da5d4da658e0306cc76f331bd2ad51f3f668 not found: ID does not exist" Sep 06 12:48:26.468330 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.468300 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-bound-sa-token\") pod \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " Sep 06 12:48:26.468453 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.468339 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-installation-pull-secrets\") pod \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " Sep 06 12:48:26.468453 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.468361 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mb9nk\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-kube-api-access-mb9nk\") pod \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " Sep 06 12:48:26.468453 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.468394 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-ca-trust-extracted\") pod \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " Sep 06 12:48:26.468453 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.468411 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls\") pod \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " Sep 06 12:48:26.468653 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.468459 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-trusted-ca\") pod \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " Sep 06 12:48:26.468653 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.468489 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-certificates\") pod \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " Sep 06 12:48:26.468653 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.468587 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-image-registry-private-configuration\") pod \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\" (UID: \"f220f886-f2be-4f33-b8fc-1d0f3dd3dafd\") " Sep 06 12:48:26.469044 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.469009 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 12:48:26.469348 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.469322 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 12:48:26.470941 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.470896 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 12:48:26.471090 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.471050 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-kube-api-access-mb9nk" (OuterVolumeSpecName: "kube-api-access-mb9nk") pod "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd"). InnerVolumeSpecName "kube-api-access-mb9nk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 12:48:26.471090 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.471069 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 12:48:26.471250 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.471211 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 12:48:26.471324 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.471244 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 12:48:26.476765 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.476741 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" (UID: "f220f886-f2be-4f33-b8fc-1d0f3dd3dafd"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 12:48:26.570192 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.570110 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-installation-pull-secrets\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:48:26.570192 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.570141 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb9nk\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-kube-api-access-mb9nk\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:48:26.570192 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.570153 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-ca-trust-extracted\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:48:26.570192 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.570165 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-tls\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:48:26.570192 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.570176 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-trusted-ca\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:48:26.570192 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.570188 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-registry-certificates\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:48:26.570489 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.570202 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-image-registry-private-configuration\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:48:26.570489 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.570214 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd-bound-sa-token\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:48:26.744640 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.744607 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6777498f4c-ccqwz"] Sep 06 12:48:26.748405 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:26.748378 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6777498f4c-ccqwz"] Sep 06 12:48:27.429124 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:27.429041 2580 generic.go:356] "Generic (PLEG): container finished" podID="b93d75ad-6183-460c-9652-c502a31ba231" containerID="b00d14d544f731c010c7664833f766cd2f2b3d8d972f1d7379c863fd7e366227" exitCode=0 Sep 06 12:48:27.429550 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:27.429118 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" event={"ID":"b93d75ad-6183-460c-9652-c502a31ba231","Type":"ContainerDied","Data":"b00d14d544f731c010c7664833f766cd2f2b3d8d972f1d7379c863fd7e366227"} Sep 06 12:48:27.429550 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:27.429521 2580 scope.go:117] "RemoveContainer" containerID="b00d14d544f731c010c7664833f766cd2f2b3d8d972f1d7379c863fd7e366227" Sep 06 12:48:28.437150 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:28.437114 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-996bm" event={"ID":"b93d75ad-6183-460c-9652-c502a31ba231","Type":"ContainerStarted","Data":"7ff854618ff681e6e5ac6443916ded16d3bf5386579748c400de43881491245f"} Sep 06 12:48:28.736576 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:28.736491 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" path="/var/lib/kubelet/pods/f220f886-f2be-4f33-b8fc-1d0f3dd3dafd/volumes" Sep 06 12:48:36.471731 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:36.471700 2580 generic.go:356] "Generic (PLEG): container finished" podID="ab6850db-dfc3-47f9-b91b-54757857d601" containerID="af4cdd27a37f614803e0e4e4fd3f3fb351007f594329c33bbf4c31105014d8c6" exitCode=0 Sep 06 12:48:36.472078 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:36.471744 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" event={"ID":"ab6850db-dfc3-47f9-b91b-54757857d601","Type":"ContainerDied","Data":"af4cdd27a37f614803e0e4e4fd3f3fb351007f594329c33bbf4c31105014d8c6"} Sep 06 12:48:36.472078 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:36.472046 2580 scope.go:117] "RemoveContainer" containerID="af4cdd27a37f614803e0e4e4fd3f3fb351007f594329c33bbf4c31105014d8c6" Sep 06 12:48:37.477551 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:37.477516 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-s6ght" event={"ID":"ab6850db-dfc3-47f9-b91b-54757857d601","Type":"ContainerStarted","Data":"6fb23fb75e35e545e71a028d61d5c37acd133e568d5c416bd47d15013bd4702d"} Sep 06 12:48:41.495380 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:41.495294 2580 generic.go:356] "Generic (PLEG): container finished" podID="fc4b28cb-5070-41be-81d6-bf4965e44af1" containerID="2da59ecb063ec6bfe8ef4b59adbcf3692931bda914e9fc42adcb5430e7bf2b7e" exitCode=0 Sep 06 12:48:41.495729 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:41.495368 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" event={"ID":"fc4b28cb-5070-41be-81d6-bf4965e44af1","Type":"ContainerDied","Data":"2da59ecb063ec6bfe8ef4b59adbcf3692931bda914e9fc42adcb5430e7bf2b7e"} Sep 06 12:48:41.495729 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:41.495705 2580 scope.go:117] "RemoveContainer" containerID="2da59ecb063ec6bfe8ef4b59adbcf3692931bda914e9fc42adcb5430e7bf2b7e" Sep 06 12:48:42.501098 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:48:42.501063 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-mw7gl" event={"ID":"fc4b28cb-5070-41be-81d6-bf4965e44af1","Type":"ContainerStarted","Data":"3f7282fb128860a42bdfc08bee654a292671694de215824847119c152d8b3df8"} Sep 06 12:51:21.515517 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.515482 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj"] Sep 06 12:51:21.515946 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.515768 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" containerName="registry" Sep 06 12:51:21.515946 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.515778 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" containerName="registry" Sep 06 12:51:21.515946 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.515843 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f220f886-f2be-4f33-b8fc-1d0f3dd3dafd" containerName="registry" Sep 06 12:51:21.518743 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.518718 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.521341 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.521254 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 06 12:51:21.521476 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.521436 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8cjd8\"" Sep 06 12:51:21.521476 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.521458 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 06 12:51:21.521597 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.521464 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 06 12:51:21.521926 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.521904 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 06 12:51:21.536946 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.536918 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj"] Sep 06 12:51:21.615389 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.615358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5db20faf-04c4-4a70-b5a3-228673e18e11-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.615389 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.615390 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5db20faf-04c4-4a70-b5a3-228673e18e11-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.615635 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.615458 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5db20faf-04c4-4a70-b5a3-228673e18e11-tmp\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.615635 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.615521 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5db20faf-04c4-4a70-b5a3-228673e18e11-manifests\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.615635 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.615555 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rkjf\" (UniqueName: \"kubernetes.io/projected/5db20faf-04c4-4a70-b5a3-228673e18e11-kube-api-access-2rkjf\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.716858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.716812 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5db20faf-04c4-4a70-b5a3-228673e18e11-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.716858 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.716857 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5db20faf-04c4-4a70-b5a3-228673e18e11-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.717109 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.716885 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5db20faf-04c4-4a70-b5a3-228673e18e11-tmp\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.717109 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.716918 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5db20faf-04c4-4a70-b5a3-228673e18e11-manifests\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.717109 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.716937 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rkjf\" (UniqueName: \"kubernetes.io/projected/5db20faf-04c4-4a70-b5a3-228673e18e11-kube-api-access-2rkjf\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.717374 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.717351 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5db20faf-04c4-4a70-b5a3-228673e18e11-manifests\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.719169 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.719146 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5db20faf-04c4-4a70-b5a3-228673e18e11-tmp\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.719416 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.719398 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5db20faf-04c4-4a70-b5a3-228673e18e11-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.719551 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.719531 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5db20faf-04c4-4a70-b5a3-228673e18e11-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.724160 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.724131 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rkjf\" (UniqueName: \"kubernetes.io/projected/5db20faf-04c4-4a70-b5a3-228673e18e11-kube-api-access-2rkjf\") pod \"opendatahub-operator-controller-manager-6b4f78958f-lqfgj\" (UID: \"5db20faf-04c4-4a70-b5a3-228673e18e11\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:21.830039 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:21.830006 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:22.108800 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:22.108727 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj"] Sep 06 12:51:22.112448 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:51:22.112417 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5db20faf_04c4_4a70_b5a3_228673e18e11.slice/crio-6da376d98179ddcb97155f1283fbe9d8f98be3dbe41f551bd73c75a0195e95aa WatchSource:0}: Error finding container 6da376d98179ddcb97155f1283fbe9d8f98be3dbe41f551bd73c75a0195e95aa: Status 404 returned error can't find the container with id 6da376d98179ddcb97155f1283fbe9d8f98be3dbe41f551bd73c75a0195e95aa Sep 06 12:51:23.109316 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:23.109251 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" event={"ID":"5db20faf-04c4-4a70-b5a3-228673e18e11","Type":"ContainerStarted","Data":"6da376d98179ddcb97155f1283fbe9d8f98be3dbe41f551bd73c75a0195e95aa"} Sep 06 12:51:25.119436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:25.119397 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" event={"ID":"5db20faf-04c4-4a70-b5a3-228673e18e11","Type":"ContainerStarted","Data":"1927836baedb7c0d38b7318856ac96b81a83eb2679af58d9d06ca5daba93def1"} Sep 06 12:51:26.125377 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:26.125345 2580 generic.go:356] "Generic (PLEG): container finished" podID="5db20faf-04c4-4a70-b5a3-228673e18e11" containerID="1927836baedb7c0d38b7318856ac96b81a83eb2679af58d9d06ca5daba93def1" exitCode=0 Sep 06 12:51:26.125801 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:26.125387 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" event={"ID":"5db20faf-04c4-4a70-b5a3-228673e18e11","Type":"ContainerDied","Data":"1927836baedb7c0d38b7318856ac96b81a83eb2679af58d9d06ca5daba93def1"} Sep 06 12:51:27.131482 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:27.131444 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" event={"ID":"5db20faf-04c4-4a70-b5a3-228673e18e11","Type":"ContainerStarted","Data":"b87fc1477764f04523fc731a76527271bd349fb03233db807a37859c6c51b3e2"} Sep 06 12:51:27.131955 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:27.131538 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:27.164367 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:27.164315 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" podStartSLOduration=1.855151301 podStartE2EDuration="6.16429913s" totalImagesPullingTime="3.26068675s" totalInitContainerRuntime="1.048461079s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:51:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:51:27.163027939 +0000 UTC m=+296.905399107" watchObservedRunningTime="2026-09-06 12:51:27.16429913 +0000 UTC m=+296.906670285" Sep 06 12:51:30.677219 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:30.677190 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 12:51:30.683080 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:30.683056 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 12:51:30.690579 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:30.690554 2580 kubelet.go:1648] "Image garbage collection succeeded" Sep 06 12:51:38.138133 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:38.138104 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b4f78958f-lqfgj" Sep 06 12:51:56.014298 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.014247 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt"] Sep 06 12:51:56.017640 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.017618 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.020707 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.020680 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 06 12:51:56.020707 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.020696 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 06 12:51:56.020893 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.020755 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 06 12:51:56.021057 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.021037 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lvjdz\"" Sep 06 12:51:56.021136 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.021101 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 06 12:51:56.021235 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.021216 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 06 12:51:56.028521 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.028500 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt"] Sep 06 12:51:56.098374 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.098329 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zqmf\" (UniqueName: \"kubernetes.io/projected/78979b32-c785-4a5e-a1ba-0a0e66320476-kube-api-access-5zqmf\") pod \"lws-controller-manager-8566865f66-tkcdt\" (UID: \"78979b32-c785-4a5e-a1ba-0a0e66320476\") " pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.098374 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.098374 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/78979b32-c785-4a5e-a1ba-0a0e66320476-manager-config\") pod \"lws-controller-manager-8566865f66-tkcdt\" (UID: \"78979b32-c785-4a5e-a1ba-0a0e66320476\") " pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.098580 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.098396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78979b32-c785-4a5e-a1ba-0a0e66320476-cert\") pod \"lws-controller-manager-8566865f66-tkcdt\" (UID: \"78979b32-c785-4a5e-a1ba-0a0e66320476\") " pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.098580 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.098488 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/78979b32-c785-4a5e-a1ba-0a0e66320476-metrics-cert\") pod \"lws-controller-manager-8566865f66-tkcdt\" (UID: \"78979b32-c785-4a5e-a1ba-0a0e66320476\") " pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.199817 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.199778 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zqmf\" (UniqueName: \"kubernetes.io/projected/78979b32-c785-4a5e-a1ba-0a0e66320476-kube-api-access-5zqmf\") pod \"lws-controller-manager-8566865f66-tkcdt\" (UID: \"78979b32-c785-4a5e-a1ba-0a0e66320476\") " pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.199817 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.199820 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/78979b32-c785-4a5e-a1ba-0a0e66320476-manager-config\") pod \"lws-controller-manager-8566865f66-tkcdt\" (UID: \"78979b32-c785-4a5e-a1ba-0a0e66320476\") " pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.200052 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.199844 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78979b32-c785-4a5e-a1ba-0a0e66320476-cert\") pod \"lws-controller-manager-8566865f66-tkcdt\" (UID: \"78979b32-c785-4a5e-a1ba-0a0e66320476\") " pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.200052 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.199974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/78979b32-c785-4a5e-a1ba-0a0e66320476-metrics-cert\") pod \"lws-controller-manager-8566865f66-tkcdt\" (UID: \"78979b32-c785-4a5e-a1ba-0a0e66320476\") " pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.200588 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.200562 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/78979b32-c785-4a5e-a1ba-0a0e66320476-manager-config\") pod \"lws-controller-manager-8566865f66-tkcdt\" (UID: \"78979b32-c785-4a5e-a1ba-0a0e66320476\") " pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.202482 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.202463 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/78979b32-c785-4a5e-a1ba-0a0e66320476-metrics-cert\") pod \"lws-controller-manager-8566865f66-tkcdt\" (UID: \"78979b32-c785-4a5e-a1ba-0a0e66320476\") " pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.202558 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.202515 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78979b32-c785-4a5e-a1ba-0a0e66320476-cert\") pod \"lws-controller-manager-8566865f66-tkcdt\" (UID: \"78979b32-c785-4a5e-a1ba-0a0e66320476\") " pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.209023 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.209001 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zqmf\" (UniqueName: \"kubernetes.io/projected/78979b32-c785-4a5e-a1ba-0a0e66320476-kube-api-access-5zqmf\") pod \"lws-controller-manager-8566865f66-tkcdt\" (UID: \"78979b32-c785-4a5e-a1ba-0a0e66320476\") " pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.326762 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.326728 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:51:56.597176 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.597147 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt"] Sep 06 12:51:56.600402 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:51:56.600376 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78979b32_c785_4a5e_a1ba_0a0e66320476.slice/crio-83e908e33a2dc8577f9def87a6021a63525b4cdbde19e7f8f2db53036a1d7de9 WatchSource:0}: Error finding container 83e908e33a2dc8577f9def87a6021a63525b4cdbde19e7f8f2db53036a1d7de9: Status 404 returned error can't find the container with id 83e908e33a2dc8577f9def87a6021a63525b4cdbde19e7f8f2db53036a1d7de9 Sep 06 12:51:56.602874 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:56.602853 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 12:51:57.255829 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:57.255791 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" event={"ID":"78979b32-c785-4a5e-a1ba-0a0e66320476","Type":"ContainerStarted","Data":"83e908e33a2dc8577f9def87a6021a63525b4cdbde19e7f8f2db53036a1d7de9"} Sep 06 12:51:59.467477 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.467440 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r"] Sep 06 12:51:59.472403 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.472381 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.475771 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.475751 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-98p7w\"" Sep 06 12:51:59.476394 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.476378 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 06 12:51:59.490988 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.490963 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r"] Sep 06 12:51:59.632817 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.632769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.633033 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.632831 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.633033 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.632885 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/930468da-ae0a-4fad-b360-6c37df34fe41-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.633033 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.632913 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/930468da-ae0a-4fad-b360-6c37df34fe41-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.633033 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.632989 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.633033 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.633011 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.633294 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.633048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/930468da-ae0a-4fad-b360-6c37df34fe41-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.633294 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.633071 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlhmd\" (UniqueName: \"kubernetes.io/projected/930468da-ae0a-4fad-b360-6c37df34fe41-kube-api-access-hlhmd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.633294 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.633095 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.734356 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.734256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/930468da-ae0a-4fad-b360-6c37df34fe41-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.734512 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.734383 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.734512 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.734415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.734512 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.734451 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/930468da-ae0a-4fad-b360-6c37df34fe41-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.734512 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.734477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlhmd\" (UniqueName: \"kubernetes.io/projected/930468da-ae0a-4fad-b360-6c37df34fe41-kube-api-access-hlhmd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.734512 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.734506 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.734780 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.734547 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.734780 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.734581 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.734780 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.734639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/930468da-ae0a-4fad-b360-6c37df34fe41-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.734913 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.734779 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.734913 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.734821 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.735079 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.735054 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/930468da-ae0a-4fad-b360-6c37df34fe41-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.735150 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.735120 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.735150 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.735140 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.737443 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.737415 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/930468da-ae0a-4fad-b360-6c37df34fe41-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.737560 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.737501 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/930468da-ae0a-4fad-b360-6c37df34fe41-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.745436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.745406 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/930468da-ae0a-4fad-b360-6c37df34fe41-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.745606 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.745579 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlhmd\" (UniqueName: \"kubernetes.io/projected/930468da-ae0a-4fad-b360-6c37df34fe41-kube-api-access-hlhmd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r\" (UID: \"930468da-ae0a-4fad-b360-6c37df34fe41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:51:59.795178 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:51:59.795144 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:52:00.075117 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:00.075093 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r"] Sep 06 12:52:00.076929 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:52:00.076893 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod930468da_ae0a_4fad_b360_6c37df34fe41.slice/crio-b9edff6eb9083fe3f217e916d32ce07b6f02724bf65369565fb5f30d647b6265 WatchSource:0}: Error finding container b9edff6eb9083fe3f217e916d32ce07b6f02724bf65369565fb5f30d647b6265: Status 404 returned error can't find the container with id b9edff6eb9083fe3f217e916d32ce07b6f02724bf65369565fb5f30d647b6265 Sep 06 12:52:00.278625 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:00.278586 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" event={"ID":"78979b32-c785-4a5e-a1ba-0a0e66320476","Type":"ContainerStarted","Data":"505394a40f26637e18cf73b12e1c92d0e3a1ee8e336c465a5d510f80a57c80e3"} Sep 06 12:52:00.278801 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:00.278736 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:52:00.279774 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:00.279750 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" event={"ID":"930468da-ae0a-4fad-b360-6c37df34fe41","Type":"ContainerStarted","Data":"b9edff6eb9083fe3f217e916d32ce07b6f02724bf65369565fb5f30d647b6265"} Sep 06 12:52:00.310313 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:00.310242 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" podStartSLOduration=2.477761684 podStartE2EDuration="5.310228589s" totalImagesPullingTime="2.832466905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:51:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:52:00.307227694 +0000 UTC m=+330.049598851" watchObservedRunningTime="2026-09-06 12:52:00.310228589 +0000 UTC m=+330.052599743" Sep 06 12:52:02.690434 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:02.690375 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 06 12:52:02.690747 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:02.690469 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 06 12:52:02.690747 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:02.690499 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 06 12:52:03.296210 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:03.296175 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" event={"ID":"930468da-ae0a-4fad-b360-6c37df34fe41","Type":"ContainerStarted","Data":"af8505bd2ae02d64a208869d13778a383af46d77dc5f2f64ad85b34d03275d80"} Sep 06 12:52:03.357236 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:03.357178 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" podStartSLOduration=1.7460755159999999 podStartE2EDuration="4.357159057s" totalImagesPullingTime="2.611083541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:51:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:52:03.354434573 +0000 UTC m=+333.096805728" watchObservedRunningTime="2026-09-06 12:52:03.357159057 +0000 UTC m=+333.099530211" Sep 06 12:52:03.795644 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:03.795612 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:52:03.800359 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:03.800338 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:52:04.301251 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:04.301215 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:52:04.302196 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:04.302174 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r" Sep 06 12:52:11.288511 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:52:11.288481 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8566865f66-tkcdt" Sep 06 12:53:09.721485 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:09.721447 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5"] Sep 06 12:53:09.723762 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:09.723746 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:09.726038 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:09.726014 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5rt92\"" Sep 06 12:53:09.726125 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:09.726014 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 06 12:53:09.726371 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:09.726355 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 06 12:53:09.736518 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:09.736496 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5"] Sep 06 12:53:09.898207 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:09.898174 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed6758d1-7cb2-41d1-8a26-659b71b2f767-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-dqjt5\" (UID: \"ed6758d1-7cb2-41d1-8a26-659b71b2f767\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:09.898403 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:09.898218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n47q\" (UniqueName: \"kubernetes.io/projected/ed6758d1-7cb2-41d1-8a26-659b71b2f767-kube-api-access-2n47q\") pod \"kuadrant-operator-controller-manager-58fc56757c-dqjt5\" (UID: \"ed6758d1-7cb2-41d1-8a26-659b71b2f767\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:09.999176 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:09.999087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed6758d1-7cb2-41d1-8a26-659b71b2f767-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-dqjt5\" (UID: \"ed6758d1-7cb2-41d1-8a26-659b71b2f767\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:09.999176 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:09.999151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n47q\" (UniqueName: \"kubernetes.io/projected/ed6758d1-7cb2-41d1-8a26-659b71b2f767-kube-api-access-2n47q\") pod \"kuadrant-operator-controller-manager-58fc56757c-dqjt5\" (UID: \"ed6758d1-7cb2-41d1-8a26-659b71b2f767\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:09.999545 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:09.999522 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed6758d1-7cb2-41d1-8a26-659b71b2f767-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-dqjt5\" (UID: \"ed6758d1-7cb2-41d1-8a26-659b71b2f767\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:10.012199 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:10.012166 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n47q\" (UniqueName: \"kubernetes.io/projected/ed6758d1-7cb2-41d1-8a26-659b71b2f767-kube-api-access-2n47q\") pod \"kuadrant-operator-controller-manager-58fc56757c-dqjt5\" (UID: \"ed6758d1-7cb2-41d1-8a26-659b71b2f767\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:10.034121 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:10.034095 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:10.310505 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:10.310482 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5"] Sep 06 12:53:10.312668 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:53:10.312632 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded6758d1_7cb2_41d1_8a26_659b71b2f767.slice/crio-62cb49e67066abd069f1f8d164e2bf7985743f05d4d1219c17eace1bdab61512 WatchSource:0}: Error finding container 62cb49e67066abd069f1f8d164e2bf7985743f05d4d1219c17eace1bdab61512: Status 404 returned error can't find the container with id 62cb49e67066abd069f1f8d164e2bf7985743f05d4d1219c17eace1bdab61512 Sep 06 12:53:10.591775 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:10.591696 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" event={"ID":"ed6758d1-7cb2-41d1-8a26-659b71b2f767","Type":"ContainerStarted","Data":"62cb49e67066abd069f1f8d164e2bf7985743f05d4d1219c17eace1bdab61512"} Sep 06 12:53:14.612515 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:14.612478 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" event={"ID":"ed6758d1-7cb2-41d1-8a26-659b71b2f767","Type":"ContainerStarted","Data":"9dd1284c76752b6a32a33fe2da54b57cda0247ffb862c42335a0bd1fb5e3083a"} Sep 06 12:53:14.612944 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:14.612576 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:14.633762 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:14.633712 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" podStartSLOduration=1.685732233 podStartE2EDuration="5.633697669s" totalImagesPullingTime="3.947965436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:53:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:53:14.631188502 +0000 UTC m=+404.373559658" watchObservedRunningTime="2026-09-06 12:53:14.633697669 +0000 UTC m=+404.376068823" Sep 06 12:53:25.619417 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:25.619378 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:27.752894 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.752856 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5"] Sep 06 12:53:27.755993 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.755972 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:27.770476 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.770451 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5"] Sep 06 12:53:27.806500 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.806469 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5"] Sep 06 12:53:27.806684 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.806650 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" podUID="ed6758d1-7cb2-41d1-8a26-659b71b2f767" containerName="manager" containerID="cri-o://9dd1284c76752b6a32a33fe2da54b57cda0247ffb862c42335a0bd1fb5e3083a" gracePeriod=2 Sep 06 12:53:27.825878 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.825850 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5"] Sep 06 12:53:27.846321 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.846293 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9653cb28-1caf-47a8-b9b7-8da09b4380bc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-hcfs5\" (UID: \"9653cb28-1caf-47a8-b9b7-8da09b4380bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:27.846437 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.846381 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zl96\" (UniqueName: \"kubernetes.io/projected/9653cb28-1caf-47a8-b9b7-8da09b4380bc-kube-api-access-5zl96\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-hcfs5\" (UID: \"9653cb28-1caf-47a8-b9b7-8da09b4380bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:27.852128 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:27.852091 2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ed6758d1-7cb2-41d1-8a26-659b71b2f767\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-dqjt5\": pods \"kuadrant-operator-controller-manager-58fc56757c-dqjt5\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:27.856814 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.856772 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5"] Sep 06 12:53:27.857122 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:27.857084 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-5zl96], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" podUID="9653cb28-1caf-47a8-b9b7-8da09b4380bc" Sep 06 12:53:27.867911 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.867433 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5"] Sep 06 12:53:27.869643 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.869617 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t"] Sep 06 12:53:27.870068 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.870050 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed6758d1-7cb2-41d1-8a26-659b71b2f767" containerName="manager" Sep 06 12:53:27.870068 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.870068 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed6758d1-7cb2-41d1-8a26-659b71b2f767" containerName="manager" Sep 06 12:53:27.870219 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.870165 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed6758d1-7cb2-41d1-8a26-659b71b2f767" containerName="manager" Sep 06 12:53:27.873143 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.873126 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" Sep 06 12:53:27.893550 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.893521 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t"] Sep 06 12:53:27.947544 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.947467 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zl96\" (UniqueName: \"kubernetes.io/projected/9653cb28-1caf-47a8-b9b7-8da09b4380bc-kube-api-access-5zl96\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-hcfs5\" (UID: \"9653cb28-1caf-47a8-b9b7-8da09b4380bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:27.948056 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.947512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c271fa42-e946-4807-bdb4-a2d7708f6feb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-xrt4t\" (UID: \"c271fa42-e946-4807-bdb4-a2d7708f6feb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" Sep 06 12:53:27.948162 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.948135 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9653cb28-1caf-47a8-b9b7-8da09b4380bc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-hcfs5\" (UID: \"9653cb28-1caf-47a8-b9b7-8da09b4380bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:27.948274 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.948237 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljk2g\" (UniqueName: \"kubernetes.io/projected/c271fa42-e946-4807-bdb4-a2d7708f6feb-kube-api-access-ljk2g\") pod \"kuadrant-operator-controller-manager-58fc56757c-xrt4t\" (UID: \"c271fa42-e946-4807-bdb4-a2d7708f6feb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" Sep 06 12:53:27.948671 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:27.948650 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9653cb28-1caf-47a8-b9b7-8da09b4380bc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-hcfs5\" (UID: \"9653cb28-1caf-47a8-b9b7-8da09b4380bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:27.950542 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:27.950507 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-dqjt5\" is forbidden: User \"system:node:ip-10-0-141-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-13.ec2.internal' and this object" podUID="ed6758d1-7cb2-41d1-8a26-659b71b2f767" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:27.953316 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:27.953292 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5zl96 for pod kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5: failed to fetch token: pod "kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" not found Sep 06 12:53:27.953450 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:27.953361 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9653cb28-1caf-47a8-b9b7-8da09b4380bc-kube-api-access-5zl96 podName:9653cb28-1caf-47a8-b9b7-8da09b4380bc nodeName:}" failed. No retries permitted until 2026-09-06 12:53:28.453345194 +0000 UTC m=+418.195716330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5zl96" (UniqueName: "kubernetes.io/projected/9653cb28-1caf-47a8-b9b7-8da09b4380bc-kube-api-access-5zl96") pod "kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" (UID: "9653cb28-1caf-47a8-b9b7-8da09b4380bc") : failed to fetch token: pod "kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" not found Sep 06 12:53:28.048919 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.048891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljk2g\" (UniqueName: \"kubernetes.io/projected/c271fa42-e946-4807-bdb4-a2d7708f6feb-kube-api-access-ljk2g\") pod \"kuadrant-operator-controller-manager-58fc56757c-xrt4t\" (UID: \"c271fa42-e946-4807-bdb4-a2d7708f6feb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" Sep 06 12:53:28.049028 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.048953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c271fa42-e946-4807-bdb4-a2d7708f6feb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-xrt4t\" (UID: \"c271fa42-e946-4807-bdb4-a2d7708f6feb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" Sep 06 12:53:28.049432 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.049296 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c271fa42-e946-4807-bdb4-a2d7708f6feb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-xrt4t\" (UID: \"c271fa42-e946-4807-bdb4-a2d7708f6feb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" Sep 06 12:53:28.057139 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.057112 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljk2g\" (UniqueName: \"kubernetes.io/projected/c271fa42-e946-4807-bdb4-a2d7708f6feb-kube-api-access-ljk2g\") pod \"kuadrant-operator-controller-manager-58fc56757c-xrt4t\" (UID: \"c271fa42-e946-4807-bdb4-a2d7708f6feb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" Sep 06 12:53:28.069929 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.069910 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:28.071768 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:28.071745 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-dqjt5\" is forbidden: User \"system:node:ip-10-0-141-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-13.ec2.internal' and this object" podUID="ed6758d1-7cb2-41d1-8a26-659b71b2f767" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:28.149549 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.149512 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2n47q\" (UniqueName: \"kubernetes.io/projected/ed6758d1-7cb2-41d1-8a26-659b71b2f767-kube-api-access-2n47q\") pod \"ed6758d1-7cb2-41d1-8a26-659b71b2f767\" (UID: \"ed6758d1-7cb2-41d1-8a26-659b71b2f767\") " Sep 06 12:53:28.149738 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.149616 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed6758d1-7cb2-41d1-8a26-659b71b2f767-extensions-socket-volume\") pod \"ed6758d1-7cb2-41d1-8a26-659b71b2f767\" (UID: \"ed6758d1-7cb2-41d1-8a26-659b71b2f767\") " Sep 06 12:53:28.150081 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.150050 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6758d1-7cb2-41d1-8a26-659b71b2f767-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ed6758d1-7cb2-41d1-8a26-659b71b2f767" (UID: "ed6758d1-7cb2-41d1-8a26-659b71b2f767"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 12:53:28.151785 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.151759 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed6758d1-7cb2-41d1-8a26-659b71b2f767-kube-api-access-2n47q" (OuterVolumeSpecName: "kube-api-access-2n47q") pod "ed6758d1-7cb2-41d1-8a26-659b71b2f767" (UID: "ed6758d1-7cb2-41d1-8a26-659b71b2f767"). InnerVolumeSpecName "kube-api-access-2n47q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 12:53:28.183970 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.183935 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" Sep 06 12:53:28.251050 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.251016 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2n47q\" (UniqueName: \"kubernetes.io/projected/ed6758d1-7cb2-41d1-8a26-659b71b2f767-kube-api-access-2n47q\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:53:28.251050 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.251047 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed6758d1-7cb2-41d1-8a26-659b71b2f767-extensions-socket-volume\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:53:28.458952 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.458837 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t"] Sep 06 12:53:28.461920 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:53:28.461895 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc271fa42_e946_4807_bdb4_a2d7708f6feb.slice/crio-6616b55372db7ddc471f3d2242d4412b204826dc17796bc21808a6820f841139 WatchSource:0}: Error finding container 6616b55372db7ddc471f3d2242d4412b204826dc17796bc21808a6820f841139: Status 404 returned error can't find the container with id 6616b55372db7ddc471f3d2242d4412b204826dc17796bc21808a6820f841139 Sep 06 12:53:28.553773 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.553739 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zl96\" (UniqueName: \"kubernetes.io/projected/9653cb28-1caf-47a8-b9b7-8da09b4380bc-kube-api-access-5zl96\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-hcfs5\" (UID: \"9653cb28-1caf-47a8-b9b7-8da09b4380bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:28.556017 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:28.555997 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5zl96 for pod kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5: failed to fetch token: pod "kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" not found Sep 06 12:53:28.556117 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:28.556060 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9653cb28-1caf-47a8-b9b7-8da09b4380bc-kube-api-access-5zl96 podName:9653cb28-1caf-47a8-b9b7-8da09b4380bc nodeName:}" failed. No retries permitted until 2026-09-06 12:53:29.556044136 +0000 UTC m=+419.298415270 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5zl96" (UniqueName: "kubernetes.io/projected/9653cb28-1caf-47a8-b9b7-8da09b4380bc-kube-api-access-5zl96") pod "kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" (UID: "9653cb28-1caf-47a8-b9b7-8da09b4380bc") : failed to fetch token: pod "kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" not found Sep 06 12:53:28.678091 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.678056 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" event={"ID":"c271fa42-e946-4807-bdb4-a2d7708f6feb","Type":"ContainerStarted","Data":"b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e"} Sep 06 12:53:28.678297 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.678112 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" event={"ID":"c271fa42-e946-4807-bdb4-a2d7708f6feb","Type":"ContainerStarted","Data":"6616b55372db7ddc471f3d2242d4412b204826dc17796bc21808a6820f841139"} Sep 06 12:53:28.678297 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.678139 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" Sep 06 12:53:28.679497 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.679467 2580 generic.go:356] "Generic (PLEG): container finished" podID="ed6758d1-7cb2-41d1-8a26-659b71b2f767" containerID="9dd1284c76752b6a32a33fe2da54b57cda0247ffb862c42335a0bd1fb5e3083a" exitCode=0 Sep 06 12:53:28.679614 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.679505 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:28.679614 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.679559 2580 scope.go:117] "RemoveContainer" containerID="9dd1284c76752b6a32a33fe2da54b57cda0247ffb862c42335a0bd1fb5e3083a" Sep 06 12:53:28.679710 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.679693 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:28.686717 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.686695 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:28.688326 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.688311 2580 scope.go:117] "RemoveContainer" containerID="9dd1284c76752b6a32a33fe2da54b57cda0247ffb862c42335a0bd1fb5e3083a" Sep 06 12:53:28.688588 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:28.688571 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dd1284c76752b6a32a33fe2da54b57cda0247ffb862c42335a0bd1fb5e3083a\": container with ID starting with 9dd1284c76752b6a32a33fe2da54b57cda0247ffb862c42335a0bd1fb5e3083a not found: ID does not exist" containerID="9dd1284c76752b6a32a33fe2da54b57cda0247ffb862c42335a0bd1fb5e3083a" Sep 06 12:53:28.688647 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.688595 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dd1284c76752b6a32a33fe2da54b57cda0247ffb862c42335a0bd1fb5e3083a"} err="failed to get container status \"9dd1284c76752b6a32a33fe2da54b57cda0247ffb862c42335a0bd1fb5e3083a\": rpc error: code = NotFound desc = could not find container \"9dd1284c76752b6a32a33fe2da54b57cda0247ffb862c42335a0bd1fb5e3083a\": container with ID starting with 9dd1284c76752b6a32a33fe2da54b57cda0247ffb862c42335a0bd1fb5e3083a not found: ID does not exist" Sep 06 12:53:28.699319 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.699248 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" podStartSLOduration=1.699234371 podStartE2EDuration="1.699234371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:53:28.697906393 +0000 UTC m=+418.440277552" watchObservedRunningTime="2026-09-06 12:53:28.699234371 +0000 UTC m=+418.441605525" Sep 06 12:53:28.699995 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:28.699969 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-dqjt5\" is forbidden: User \"system:node:ip-10-0-141-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-13.ec2.internal' and this object" podUID="ed6758d1-7cb2-41d1-8a26-659b71b2f767" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:28.701631 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:28.701608 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-dqjt5\" is forbidden: User \"system:node:ip-10-0-141-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-13.ec2.internal' and this object" podUID="ed6758d1-7cb2-41d1-8a26-659b71b2f767" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-dqjt5" Sep 06 12:53:28.703143 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:28.703117 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5b65f94d4c-hcfs5\" is forbidden: User \"system:node:ip-10-0-141-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-13.ec2.internal' and this object" podUID="9653cb28-1caf-47a8-b9b7-8da09b4380bc" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:28.735714 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.735638 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed6758d1-7cb2-41d1-8a26-659b71b2f767" path="/var/lib/kubelet/pods/ed6758d1-7cb2-41d1-8a26-659b71b2f767/volumes" Sep 06 12:53:28.755835 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.755808 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9653cb28-1caf-47a8-b9b7-8da09b4380bc-extensions-socket-volume\") pod \"9653cb28-1caf-47a8-b9b7-8da09b4380bc\" (UID: \"9653cb28-1caf-47a8-b9b7-8da09b4380bc\") " Sep 06 12:53:28.756174 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.756087 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9653cb28-1caf-47a8-b9b7-8da09b4380bc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9653cb28-1caf-47a8-b9b7-8da09b4380bc" (UID: "9653cb28-1caf-47a8-b9b7-8da09b4380bc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 12:53:28.756174 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.756114 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zl96\" (UniqueName: \"kubernetes.io/projected/9653cb28-1caf-47a8-b9b7-8da09b4380bc-kube-api-access-5zl96\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:53:28.856617 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:28.856574 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9653cb28-1caf-47a8-b9b7-8da09b4380bc-extensions-socket-volume\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:53:29.684809 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:29.684779 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:29.693119 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:29.693093 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5b65f94d4c-hcfs5\" is forbidden: User \"system:node:ip-10-0-141-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-13.ec2.internal' and this object" podUID="9653cb28-1caf-47a8-b9b7-8da09b4380bc" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:30.736973 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:30.736933 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5b65f94d4c-hcfs5\" is forbidden: User \"system:node:ip-10-0-141-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-13.ec2.internal' and this object" podUID="9653cb28-1caf-47a8-b9b7-8da09b4380bc" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-hcfs5" Sep 06 12:53:30.740442 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:30.740413 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9653cb28-1caf-47a8-b9b7-8da09b4380bc" path="/var/lib/kubelet/pods/9653cb28-1caf-47a8-b9b7-8da09b4380bc/volumes" Sep 06 12:53:39.686930 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:39.686895 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" Sep 06 12:53:43.689222 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:43.689189 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t"] Sep 06 12:53:43.689777 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:43.689389 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" podUID="c271fa42-e946-4807-bdb4-a2d7708f6feb" containerName="manager" containerID="cri-o://b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e" gracePeriod=10 Sep 06 12:53:43.930524 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:43.930500 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" Sep 06 12:53:43.987419 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:43.987335 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c271fa42-e946-4807-bdb4-a2d7708f6feb-extensions-socket-volume\") pod \"c271fa42-e946-4807-bdb4-a2d7708f6feb\" (UID: \"c271fa42-e946-4807-bdb4-a2d7708f6feb\") " Sep 06 12:53:43.987587 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:43.987433 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljk2g\" (UniqueName: \"kubernetes.io/projected/c271fa42-e946-4807-bdb4-a2d7708f6feb-kube-api-access-ljk2g\") pod \"c271fa42-e946-4807-bdb4-a2d7708f6feb\" (UID: \"c271fa42-e946-4807-bdb4-a2d7708f6feb\") " Sep 06 12:53:43.987753 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:43.987724 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c271fa42-e946-4807-bdb4-a2d7708f6feb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c271fa42-e946-4807-bdb4-a2d7708f6feb" (UID: "c271fa42-e946-4807-bdb4-a2d7708f6feb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 12:53:43.989444 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:43.989422 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c271fa42-e946-4807-bdb4-a2d7708f6feb-kube-api-access-ljk2g" (OuterVolumeSpecName: "kube-api-access-ljk2g") pod "c271fa42-e946-4807-bdb4-a2d7708f6feb" (UID: "c271fa42-e946-4807-bdb4-a2d7708f6feb"). InnerVolumeSpecName "kube-api-access-ljk2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 12:53:44.088231 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:44.088195 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljk2g\" (UniqueName: \"kubernetes.io/projected/c271fa42-e946-4807-bdb4-a2d7708f6feb-kube-api-access-ljk2g\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:53:44.088231 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:44.088224 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c271fa42-e946-4807-bdb4-a2d7708f6feb-extensions-socket-volume\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:53:44.752620 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:44.752590 2580 generic.go:356] "Generic (PLEG): container finished" podID="c271fa42-e946-4807-bdb4-a2d7708f6feb" containerID="b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e" exitCode=0 Sep 06 12:53:44.753007 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:44.752657 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" Sep 06 12:53:44.753007 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:44.752674 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" event={"ID":"c271fa42-e946-4807-bdb4-a2d7708f6feb","Type":"ContainerDied","Data":"b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e"} Sep 06 12:53:44.753007 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:44.752729 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t" event={"ID":"c271fa42-e946-4807-bdb4-a2d7708f6feb","Type":"ContainerDied","Data":"6616b55372db7ddc471f3d2242d4412b204826dc17796bc21808a6820f841139"} Sep 06 12:53:44.753007 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:44.752751 2580 scope.go:117] "RemoveContainer" containerID="b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e" Sep 06 12:53:44.760962 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:44.760945 2580 scope.go:117] "RemoveContainer" containerID="b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e" Sep 06 12:53:44.761179 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:53:44.761161 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e\": container with ID starting with b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e not found: ID does not exist" containerID="b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e" Sep 06 12:53:44.761223 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:44.761187 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e"} err="failed to get container status \"b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e\": rpc error: code = NotFound desc = could not find container \"b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e\": container with ID starting with b1f3f76a60455c07f10c7c5de9f1a36926f10ef67a677839a5a69d03609d881e not found: ID does not exist" Sep 06 12:53:44.779019 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:44.778996 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t"] Sep 06 12:53:44.793111 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:44.793091 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-xrt4t"] Sep 06 12:53:46.735758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:46.735723 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c271fa42-e946-4807-bdb4-a2d7708f6feb" path="/var/lib/kubelet/pods/c271fa42-e946-4807-bdb4-a2d7708f6feb/volumes" Sep 06 12:53:59.952742 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:59.952657 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m"] Sep 06 12:53:59.953182 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:59.953161 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c271fa42-e946-4807-bdb4-a2d7708f6feb" containerName="manager" Sep 06 12:53:59.953232 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:59.953188 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c271fa42-e946-4807-bdb4-a2d7708f6feb" containerName="manager" Sep 06 12:53:59.953330 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:59.953316 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c271fa42-e946-4807-bdb4-a2d7708f6feb" containerName="manager" Sep 06 12:53:59.955682 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:59.955663 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:53:59.957860 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:59.957834 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8msjv\"" Sep 06 12:53:59.967362 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:53:59.967335 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m"] Sep 06 12:54:00.129639 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.129592 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.129639 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.129640 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckk2b\" (UniqueName: \"kubernetes.io/projected/5178826f-8d7f-4be4-871b-528202e9198d-kube-api-access-ckk2b\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.129906 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.129744 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.129906 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.129789 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5178826f-8d7f-4be4-871b-528202e9198d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.129906 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.129834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.129906 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.129873 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.130061 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.129911 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5178826f-8d7f-4be4-871b-528202e9198d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.130061 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.129939 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.130061 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.129965 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5178826f-8d7f-4be4-871b-528202e9198d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.230617 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.230528 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckk2b\" (UniqueName: \"kubernetes.io/projected/5178826f-8d7f-4be4-871b-528202e9198d-kube-api-access-ckk2b\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.230617 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.230590 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.230850 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.230621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5178826f-8d7f-4be4-871b-528202e9198d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.230850 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.230648 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.230850 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.230737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.230850 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.230818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5178826f-8d7f-4be4-871b-528202e9198d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.231126 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.230847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.231223 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.231202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5178826f-8d7f-4be4-871b-528202e9198d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.231346 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.231317 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.231483 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.231204 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.231483 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.231095 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.231483 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.231103 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.231672 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.231648 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.231733 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.231709 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5178826f-8d7f-4be4-871b-528202e9198d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.233149 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.233128 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5178826f-8d7f-4be4-871b-528202e9198d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.233505 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.233486 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5178826f-8d7f-4be4-871b-528202e9198d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.237926 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.237896 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5178826f-8d7f-4be4-871b-528202e9198d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.238100 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.238083 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckk2b\" (UniqueName: \"kubernetes.io/projected/5178826f-8d7f-4be4-871b-528202e9198d-kube-api-access-ckk2b\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6gs8m\" (UID: \"5178826f-8d7f-4be4-871b-528202e9198d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.267736 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.267705 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:00.544163 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.544137 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m"] Sep 06 12:54:00.546683 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:54:00.546657 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5178826f_8d7f_4be4_871b_528202e9198d.slice/crio-9b7cd596c3ea64325d4156a50a3de32bb3892b27c1b01e74690ad4dc61d71edc WatchSource:0}: Error finding container 9b7cd596c3ea64325d4156a50a3de32bb3892b27c1b01e74690ad4dc61d71edc: Status 404 returned error can't find the container with id 9b7cd596c3ea64325d4156a50a3de32bb3892b27c1b01e74690ad4dc61d71edc Sep 06 12:54:00.549307 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.549244 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 06 12:54:00.549410 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.549347 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 06 12:54:00.549410 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.549377 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 06 12:54:00.822597 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.822560 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" event={"ID":"5178826f-8d7f-4be4-871b-528202e9198d","Type":"ContainerStarted","Data":"c1c76e7287ddf2df77f5d3ec5c163fbe9940abc9ba3014d3195dc3c50079addb"} Sep 06 12:54:00.822754 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.822628 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" event={"ID":"5178826f-8d7f-4be4-871b-528202e9198d","Type":"ContainerStarted","Data":"9b7cd596c3ea64325d4156a50a3de32bb3892b27c1b01e74690ad4dc61d71edc"} Sep 06 12:54:00.844056 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:00.843998 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" podStartSLOduration=1.843976981 podStartE2EDuration="1.843976981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:53:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:54:00.840944198 +0000 UTC m=+450.583315352" watchObservedRunningTime="2026-09-06 12:54:00.843976981 +0000 UTC m=+450.586348138" Sep 06 12:54:01.268387 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.268296 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:01.273215 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.273189 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:01.801118 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.801086 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-kgsmm"] Sep 06 12:54:01.803435 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.803419 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" Sep 06 12:54:01.805754 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.805727 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 06 12:54:01.806479 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.806459 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 06 12:54:01.806565 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.806459 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 06 12:54:01.806565 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.806499 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-n82sk\"" Sep 06 12:54:01.815634 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.815566 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-kgsmm"] Sep 06 12:54:01.828323 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.828290 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:01.829399 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.829375 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:54:01.897532 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.897495 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-kgsmm"] Sep 06 12:54:01.946083 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.946048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bc4a7d0-7501-48dd-b1ba-65041aaa5165-config-file\") pod \"limitador-limitador-859c4d96d6-kgsmm\" (UID: \"9bc4a7d0-7501-48dd-b1ba-65041aaa5165\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" Sep 06 12:54:01.946280 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:01.946162 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnhsj\" (UniqueName: \"kubernetes.io/projected/9bc4a7d0-7501-48dd-b1ba-65041aaa5165-kube-api-access-nnhsj\") pod \"limitador-limitador-859c4d96d6-kgsmm\" (UID: \"9bc4a7d0-7501-48dd-b1ba-65041aaa5165\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" Sep 06 12:54:02.047040 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.046999 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bc4a7d0-7501-48dd-b1ba-65041aaa5165-config-file\") pod \"limitador-limitador-859c4d96d6-kgsmm\" (UID: \"9bc4a7d0-7501-48dd-b1ba-65041aaa5165\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" Sep 06 12:54:02.047243 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.047097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnhsj\" (UniqueName: \"kubernetes.io/projected/9bc4a7d0-7501-48dd-b1ba-65041aaa5165-kube-api-access-nnhsj\") pod \"limitador-limitador-859c4d96d6-kgsmm\" (UID: \"9bc4a7d0-7501-48dd-b1ba-65041aaa5165\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" Sep 06 12:54:02.047756 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.047728 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bc4a7d0-7501-48dd-b1ba-65041aaa5165-config-file\") pod \"limitador-limitador-859c4d96d6-kgsmm\" (UID: \"9bc4a7d0-7501-48dd-b1ba-65041aaa5165\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" Sep 06 12:54:02.054420 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.054361 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnhsj\" (UniqueName: \"kubernetes.io/projected/9bc4a7d0-7501-48dd-b1ba-65041aaa5165-kube-api-access-nnhsj\") pod \"limitador-limitador-859c4d96d6-kgsmm\" (UID: \"9bc4a7d0-7501-48dd-b1ba-65041aaa5165\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" Sep 06 12:54:02.113700 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.113661 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" Sep 06 12:54:02.383203 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.383175 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-kgsmm"] Sep 06 12:54:02.385429 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:54:02.385392 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc4a7d0_7501_48dd_b1ba_65041aaa5165.slice/crio-649c5be85dc9bb5d72910b60feebf44940ec637a0796c5bb5ce4e59ca7320e09 WatchSource:0}: Error finding container 649c5be85dc9bb5d72910b60feebf44940ec637a0796c5bb5ce4e59ca7320e09: Status 404 returned error can't find the container with id 649c5be85dc9bb5d72910b60feebf44940ec637a0796c5bb5ce4e59ca7320e09 Sep 06 12:54:02.595371 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.595334 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-8lvgs"] Sep 06 12:54:02.598440 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.598416 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-8lvgs" Sep 06 12:54:02.600522 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.600498 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-j9jkp\"" Sep 06 12:54:02.607415 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.607386 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-8lvgs"] Sep 06 12:54:02.753889 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.753797 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76ckz\" (UniqueName: \"kubernetes.io/projected/7c4cb728-14de-4538-b1e8-7e54dd6290a5-kube-api-access-76ckz\") pod \"authorino-b54df94b4-8lvgs\" (UID: \"7c4cb728-14de-4538-b1e8-7e54dd6290a5\") " pod="kuadrant-system/authorino-b54df94b4-8lvgs" Sep 06 12:54:02.834150 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.834112 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" event={"ID":"9bc4a7d0-7501-48dd-b1ba-65041aaa5165","Type":"ContainerStarted","Data":"649c5be85dc9bb5d72910b60feebf44940ec637a0796c5bb5ce4e59ca7320e09"} Sep 06 12:54:02.855049 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.855004 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76ckz\" (UniqueName: \"kubernetes.io/projected/7c4cb728-14de-4538-b1e8-7e54dd6290a5-kube-api-access-76ckz\") pod \"authorino-b54df94b4-8lvgs\" (UID: \"7c4cb728-14de-4538-b1e8-7e54dd6290a5\") " pod="kuadrant-system/authorino-b54df94b4-8lvgs" Sep 06 12:54:02.864027 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.863980 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76ckz\" (UniqueName: \"kubernetes.io/projected/7c4cb728-14de-4538-b1e8-7e54dd6290a5-kube-api-access-76ckz\") pod \"authorino-b54df94b4-8lvgs\" (UID: \"7c4cb728-14de-4538-b1e8-7e54dd6290a5\") " pod="kuadrant-system/authorino-b54df94b4-8lvgs" Sep 06 12:54:02.908864 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:02.908826 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-8lvgs" Sep 06 12:54:03.176953 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:03.176921 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-8lvgs"] Sep 06 12:54:03.180280 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:54:03.180234 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c4cb728_14de_4538_b1e8_7e54dd6290a5.slice/crio-5976ec9b4a5d758783a761b9e1e86b226cc2118137db46dfc66678c112679d91 WatchSource:0}: Error finding container 5976ec9b4a5d758783a761b9e1e86b226cc2118137db46dfc66678c112679d91: Status 404 returned error can't find the container with id 5976ec9b4a5d758783a761b9e1e86b226cc2118137db46dfc66678c112679d91 Sep 06 12:54:03.840103 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:03.840067 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-8lvgs" event={"ID":"7c4cb728-14de-4538-b1e8-7e54dd6290a5","Type":"ContainerStarted","Data":"5976ec9b4a5d758783a761b9e1e86b226cc2118137db46dfc66678c112679d91"} Sep 06 12:54:03.841522 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:03.841493 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" event={"ID":"9bc4a7d0-7501-48dd-b1ba-65041aaa5165","Type":"ContainerStarted","Data":"93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0"} Sep 06 12:54:03.858606 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:03.858555 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" podStartSLOduration=1.493467448 podStartE2EDuration="2.858538446s" totalImagesPullingTime="1.365070998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:54:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:54:03.857322888 +0000 UTC m=+453.599694044" watchObservedRunningTime="2026-09-06 12:54:03.858538446 +0000 UTC m=+453.600909611" Sep 06 12:54:04.847244 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:04.847209 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" Sep 06 12:54:05.697724 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:05.697688 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-8lvgs"] Sep 06 12:54:05.852769 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:05.852729 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-8lvgs" event={"ID":"7c4cb728-14de-4538-b1e8-7e54dd6290a5","Type":"ContainerStarted","Data":"ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649"} Sep 06 12:54:05.867159 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:05.867105 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-8lvgs" podStartSLOduration=1.9347156989999998 podStartE2EDuration="3.867086331s" totalImagesPullingTime="1.932370632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:54:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:54:05.865757358 +0000 UTC m=+455.608128514" watchObservedRunningTime="2026-09-06 12:54:05.867086331 +0000 UTC m=+455.609457485" Sep 06 12:54:06.857100 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:06.857062 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-8lvgs" podUID="7c4cb728-14de-4538-b1e8-7e54dd6290a5" containerName="authorino" containerID="cri-o://ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649" gracePeriod=30 Sep 06 12:54:07.101925 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.101900 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-8lvgs" Sep 06 12:54:07.191374 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.191258 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76ckz\" (UniqueName: \"kubernetes.io/projected/7c4cb728-14de-4538-b1e8-7e54dd6290a5-kube-api-access-76ckz\") pod \"7c4cb728-14de-4538-b1e8-7e54dd6290a5\" (UID: \"7c4cb728-14de-4538-b1e8-7e54dd6290a5\") " Sep 06 12:54:07.193333 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.193302 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c4cb728-14de-4538-b1e8-7e54dd6290a5-kube-api-access-76ckz" (OuterVolumeSpecName: "kube-api-access-76ckz") pod "7c4cb728-14de-4538-b1e8-7e54dd6290a5" (UID: "7c4cb728-14de-4538-b1e8-7e54dd6290a5"). InnerVolumeSpecName "kube-api-access-76ckz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 12:54:07.292412 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.292381 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76ckz\" (UniqueName: \"kubernetes.io/projected/7c4cb728-14de-4538-b1e8-7e54dd6290a5-kube-api-access-76ckz\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:54:07.863827 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.863792 2580 generic.go:356] "Generic (PLEG): container finished" podID="7c4cb728-14de-4538-b1e8-7e54dd6290a5" containerID="ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649" exitCode=0 Sep 06 12:54:07.864291 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.863846 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-8lvgs" Sep 06 12:54:07.864291 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.863875 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-8lvgs" event={"ID":"7c4cb728-14de-4538-b1e8-7e54dd6290a5","Type":"ContainerDied","Data":"ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649"} Sep 06 12:54:07.864291 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.863914 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-8lvgs" event={"ID":"7c4cb728-14de-4538-b1e8-7e54dd6290a5","Type":"ContainerDied","Data":"5976ec9b4a5d758783a761b9e1e86b226cc2118137db46dfc66678c112679d91"} Sep 06 12:54:07.864291 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.863937 2580 scope.go:117] "RemoveContainer" containerID="ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649" Sep 06 12:54:07.873002 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.872986 2580 scope.go:117] "RemoveContainer" containerID="ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649" Sep 06 12:54:07.873280 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:54:07.873246 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649\": container with ID starting with ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649 not found: ID does not exist" containerID="ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649" Sep 06 12:54:07.873338 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.873283 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649"} err="failed to get container status \"ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649\": rpc error: code = NotFound desc = could not find container \"ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649\": container with ID starting with ae63b029e1b7df42e83ad72c21ed87f0b456592aed5756f6aee60c3e478ed649 not found: ID does not exist" Sep 06 12:54:07.885616 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.885590 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-8lvgs"] Sep 06 12:54:07.889220 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:07.889198 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-8lvgs"] Sep 06 12:54:08.737551 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:08.737508 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c4cb728-14de-4538-b1e8-7e54dd6290a5" path="/var/lib/kubelet/pods/7c4cb728-14de-4538-b1e8-7e54dd6290a5/volumes" Sep 06 12:54:15.854013 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:15.853977 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" Sep 06 12:54:15.858419 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:15.858391 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-kgsmm"] Sep 06 12:54:15.899838 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:15.899775 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" podUID="9bc4a7d0-7501-48dd-b1ba-65041aaa5165" containerName="limitador" containerID="cri-o://93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0" gracePeriod=30 Sep 06 12:54:16.438596 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.438573 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" Sep 06 12:54:16.471281 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.471199 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnhsj\" (UniqueName: \"kubernetes.io/projected/9bc4a7d0-7501-48dd-b1ba-65041aaa5165-kube-api-access-nnhsj\") pod \"9bc4a7d0-7501-48dd-b1ba-65041aaa5165\" (UID: \"9bc4a7d0-7501-48dd-b1ba-65041aaa5165\") " Sep 06 12:54:16.471281 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.471243 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bc4a7d0-7501-48dd-b1ba-65041aaa5165-config-file\") pod \"9bc4a7d0-7501-48dd-b1ba-65041aaa5165\" (UID: \"9bc4a7d0-7501-48dd-b1ba-65041aaa5165\") " Sep 06 12:54:16.471721 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.471695 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bc4a7d0-7501-48dd-b1ba-65041aaa5165-config-file" (OuterVolumeSpecName: "config-file") pod "9bc4a7d0-7501-48dd-b1ba-65041aaa5165" (UID: "9bc4a7d0-7501-48dd-b1ba-65041aaa5165"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 12:54:16.473173 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.473152 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bc4a7d0-7501-48dd-b1ba-65041aaa5165-kube-api-access-nnhsj" (OuterVolumeSpecName: "kube-api-access-nnhsj") pod "9bc4a7d0-7501-48dd-b1ba-65041aaa5165" (UID: "9bc4a7d0-7501-48dd-b1ba-65041aaa5165"). InnerVolumeSpecName "kube-api-access-nnhsj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 12:54:16.572245 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.572209 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnhsj\" (UniqueName: \"kubernetes.io/projected/9bc4a7d0-7501-48dd-b1ba-65041aaa5165-kube-api-access-nnhsj\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:54:16.572245 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.572239 2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bc4a7d0-7501-48dd-b1ba-65041aaa5165-config-file\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:54:16.904986 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.904949 2580 generic.go:356] "Generic (PLEG): container finished" podID="9bc4a7d0-7501-48dd-b1ba-65041aaa5165" containerID="93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0" exitCode=0 Sep 06 12:54:16.905482 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.904995 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" Sep 06 12:54:16.905482 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.905035 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" event={"ID":"9bc4a7d0-7501-48dd-b1ba-65041aaa5165","Type":"ContainerDied","Data":"93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0"} Sep 06 12:54:16.905482 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.905080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-kgsmm" event={"ID":"9bc4a7d0-7501-48dd-b1ba-65041aaa5165","Type":"ContainerDied","Data":"649c5be85dc9bb5d72910b60feebf44940ec637a0796c5bb5ce4e59ca7320e09"} Sep 06 12:54:16.905482 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.905103 2580 scope.go:117] "RemoveContainer" containerID="93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0" Sep 06 12:54:16.913375 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.913357 2580 scope.go:117] "RemoveContainer" containerID="93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0" Sep 06 12:54:16.913627 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:54:16.913605 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0\": container with ID starting with 93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0 not found: ID does not exist" containerID="93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0" Sep 06 12:54:16.913686 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.913634 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0"} err="failed to get container status \"93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0\": rpc error: code = NotFound desc = could not find container \"93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0\": container with ID starting with 93f657e5cae294376ef09f104fd53ce029afe5bf8221cb9d2ee17e085b361ad0 not found: ID does not exist" Sep 06 12:54:16.920244 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.920221 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-kgsmm"] Sep 06 12:54:16.924204 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:16.924184 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-kgsmm"] Sep 06 12:54:18.345721 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.345688 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-j4862"] Sep 06 12:54:18.346422 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.346397 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c4cb728-14de-4538-b1e8-7e54dd6290a5" containerName="authorino" Sep 06 12:54:18.346626 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.346601 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c4cb728-14de-4538-b1e8-7e54dd6290a5" containerName="authorino" Sep 06 12:54:18.346698 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.346669 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc4a7d0-7501-48dd-b1ba-65041aaa5165" containerName="limitador" Sep 06 12:54:18.346698 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.346680 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc4a7d0-7501-48dd-b1ba-65041aaa5165" containerName="limitador" Sep 06 12:54:18.346981 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.346966 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bc4a7d0-7501-48dd-b1ba-65041aaa5165" containerName="limitador" Sep 06 12:54:18.347048 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.347001 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c4cb728-14de-4538-b1e8-7e54dd6290a5" containerName="authorino" Sep 06 12:54:18.352284 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.352243 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" Sep 06 12:54:18.355765 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.355747 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 06 12:54:18.357619 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.357588 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 06 12:54:18.357722 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.357676 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 12:54:18.360742 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.360723 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-74f6n\"" Sep 06 12:54:18.373653 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.373630 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-j4862"] Sep 06 12:54:18.387236 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.387210 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f43fa617-d08a-437f-b0b5-833bcbcda082-data\") pod \"postgres-69fb4594cf-j4862\" (UID: \"f43fa617-d08a-437f-b0b5-833bcbcda082\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" Sep 06 12:54:18.387337 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.387252 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bcr9\" (UniqueName: \"kubernetes.io/projected/f43fa617-d08a-437f-b0b5-833bcbcda082-kube-api-access-7bcr9\") pod \"postgres-69fb4594cf-j4862\" (UID: \"f43fa617-d08a-437f-b0b5-833bcbcda082\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" Sep 06 12:54:18.488248 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.488211 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f43fa617-d08a-437f-b0b5-833bcbcda082-data\") pod \"postgres-69fb4594cf-j4862\" (UID: \"f43fa617-d08a-437f-b0b5-833bcbcda082\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" Sep 06 12:54:18.488432 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.488291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bcr9\" (UniqueName: \"kubernetes.io/projected/f43fa617-d08a-437f-b0b5-833bcbcda082-kube-api-access-7bcr9\") pod \"postgres-69fb4594cf-j4862\" (UID: \"f43fa617-d08a-437f-b0b5-833bcbcda082\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" Sep 06 12:54:18.488634 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.488616 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f43fa617-d08a-437f-b0b5-833bcbcda082-data\") pod \"postgres-69fb4594cf-j4862\" (UID: \"f43fa617-d08a-437f-b0b5-833bcbcda082\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" Sep 06 12:54:18.498402 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.498379 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bcr9\" (UniqueName: \"kubernetes.io/projected/f43fa617-d08a-437f-b0b5-833bcbcda082-kube-api-access-7bcr9\") pod \"postgres-69fb4594cf-j4862\" (UID: \"f43fa617-d08a-437f-b0b5-833bcbcda082\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" Sep 06 12:54:18.662606 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.662526 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" Sep 06 12:54:18.737582 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.737544 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bc4a7d0-7501-48dd-b1ba-65041aaa5165" path="/var/lib/kubelet/pods/9bc4a7d0-7501-48dd-b1ba-65041aaa5165/volumes" Sep 06 12:54:18.932639 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:18.932615 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-j4862"] Sep 06 12:54:18.935069 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:54:18.935044 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf43fa617_d08a_437f_b0b5_833bcbcda082.slice/crio-cea2fb0001924c034ab57bb459949fe17bc84fd897ed5fa125a5405d35de4047 WatchSource:0}: Error finding container cea2fb0001924c034ab57bb459949fe17bc84fd897ed5fa125a5405d35de4047: Status 404 returned error can't find the container with id cea2fb0001924c034ab57bb459949fe17bc84fd897ed5fa125a5405d35de4047 Sep 06 12:54:19.925436 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:19.925351 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" event={"ID":"f43fa617-d08a-437f-b0b5-833bcbcda082","Type":"ContainerStarted","Data":"cea2fb0001924c034ab57bb459949fe17bc84fd897ed5fa125a5405d35de4047"} Sep 06 12:54:23.945639 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:23.945539 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" event={"ID":"f43fa617-d08a-437f-b0b5-833bcbcda082","Type":"ContainerStarted","Data":"915aa6e1da7c4c13e81b11313211427ff3bed2a33c9926402bbb372b237d0217"} Sep 06 12:54:23.945639 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:23.945630 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" Sep 06 12:54:23.960952 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:23.960898 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" podStartSLOduration=1.25437859 podStartE2EDuration="5.960881838s" totalImagesPullingTime="4.706503248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:54:23.959117125 +0000 UTC m=+473.701488280" watchObservedRunningTime="2026-09-06 12:54:23.960881838 +0000 UTC m=+473.703252993" Sep 06 12:54:29.979494 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:29.979462 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4862" Sep 06 12:54:30.518608 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.518575 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-w9smq"] Sep 06 12:54:30.524115 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.524096 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-w9smq" Sep 06 12:54:30.526531 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.526508 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-j9jkp\"" Sep 06 12:54:30.526672 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.526534 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 06 12:54:30.526672 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.526547 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 06 12:54:30.526672 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.526585 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-w9smq"] Sep 06 12:54:30.576086 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.576052 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-w9smq"] Sep 06 12:54:30.576346 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:54:30.576325 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-dv4lf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-w9smq" podUID="f74fbffb-e347-46ce-b43e-26d35a22c731" Sep 06 12:54:30.602084 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.602051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv4lf\" (UniqueName: \"kubernetes.io/projected/f74fbffb-e347-46ce-b43e-26d35a22c731-kube-api-access-dv4lf\") pod \"authorino-5dc7ccb5f-w9smq\" (UID: \"f74fbffb-e347-46ce-b43e-26d35a22c731\") " pod="kuadrant-system/authorino-5dc7ccb5f-w9smq" Sep 06 12:54:30.703203 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.703171 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv4lf\" (UniqueName: \"kubernetes.io/projected/f74fbffb-e347-46ce-b43e-26d35a22c731-kube-api-access-dv4lf\") pod \"authorino-5dc7ccb5f-w9smq\" (UID: \"f74fbffb-e347-46ce-b43e-26d35a22c731\") " pod="kuadrant-system/authorino-5dc7ccb5f-w9smq" Sep 06 12:54:30.710553 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.710535 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 06 12:54:30.721594 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.721562 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 06 12:54:30.732472 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.732449 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv4lf\" (UniqueName: \"kubernetes.io/projected/f74fbffb-e347-46ce-b43e-26d35a22c731-kube-api-access-dv4lf\") pod \"authorino-5dc7ccb5f-w9smq\" (UID: \"f74fbffb-e347-46ce-b43e-26d35a22c731\") " pod="kuadrant-system/authorino-5dc7ccb5f-w9smq" Sep 06 12:54:30.978736 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.978706 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-w9smq" Sep 06 12:54:30.987750 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:30.987727 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-w9smq" Sep 06 12:54:31.106812 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:31.106783 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dv4lf\" (UniqueName: \"kubernetes.io/projected/f74fbffb-e347-46ce-b43e-26d35a22c731-kube-api-access-dv4lf\") pod \"f74fbffb-e347-46ce-b43e-26d35a22c731\" (UID: \"f74fbffb-e347-46ce-b43e-26d35a22c731\") " Sep 06 12:54:31.108856 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:31.108825 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f74fbffb-e347-46ce-b43e-26d35a22c731-kube-api-access-dv4lf" (OuterVolumeSpecName: "kube-api-access-dv4lf") pod "f74fbffb-e347-46ce-b43e-26d35a22c731" (UID: "f74fbffb-e347-46ce-b43e-26d35a22c731"). InnerVolumeSpecName "kube-api-access-dv4lf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 12:54:31.208302 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:31.208249 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dv4lf\" (UniqueName: \"kubernetes.io/projected/f74fbffb-e347-46ce-b43e-26d35a22c731-kube-api-access-dv4lf\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:54:31.983376 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:31.983338 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-w9smq" Sep 06 12:54:32.020070 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:32.020039 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-w9smq"] Sep 06 12:54:32.027316 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:32.027287 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-w9smq"] Sep 06 12:54:32.735799 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:32.735764 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f74fbffb-e347-46ce-b43e-26d35a22c731" path="/var/lib/kubelet/pods/f74fbffb-e347-46ce-b43e-26d35a22c731/volumes" Sep 06 12:54:38.451306 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.451242 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74d564bf49-s94c2"] Sep 06 12:54:38.466489 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.466461 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:38.469564 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.469541 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rdbzn\"" Sep 06 12:54:38.469862 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.469842 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 06 12:54:38.470246 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.470227 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 06 12:54:38.470364 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.470227 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 06 12:54:38.479747 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.479722 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74d564bf49-s94c2"] Sep 06 12:54:38.571356 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.571312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-metrics-certs\") pod \"maas-controller-74d564bf49-s94c2\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:38.571536 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.571419 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-webhook-cert\") pod \"maas-controller-74d564bf49-s94c2\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:38.571536 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.571449 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fbnk\" (UniqueName: \"kubernetes.io/projected/0e9661e6-83b5-4cc7-b0e5-55de73661d99-kube-api-access-8fbnk\") pod \"maas-controller-74d564bf49-s94c2\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:38.635786 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.635760 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-777d65df8-f4lkl"] Sep 06 12:54:38.637941 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.637918 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:38.649739 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.649713 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-777d65df8-f4lkl"] Sep 06 12:54:38.672289 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.672235 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-metrics-certs\") pod \"maas-controller-74d564bf49-s94c2\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:38.672418 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.672326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-webhook-cert\") pod \"maas-controller-74d564bf49-s94c2\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:38.672418 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.672356 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fbnk\" (UniqueName: \"kubernetes.io/projected/0e9661e6-83b5-4cc7-b0e5-55de73661d99-kube-api-access-8fbnk\") pod \"maas-controller-74d564bf49-s94c2\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:38.672418 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:54:38.672390 2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 06 12:54:38.672546 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:54:38.672468 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-metrics-certs podName:0e9661e6-83b5-4cc7-b0e5-55de73661d99 nodeName:}" failed. No retries permitted until 2026-09-06 12:54:39.172447104 +0000 UTC m=+488.914818251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-metrics-certs") pod "maas-controller-74d564bf49-s94c2" (UID: "0e9661e6-83b5-4cc7-b0e5-55de73661d99") : secret "maas-controller-metrics-tls" not found Sep 06 12:54:38.672546 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:54:38.672474 2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 06 12:54:38.672546 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:54:38.672525 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-webhook-cert podName:0e9661e6-83b5-4cc7-b0e5-55de73661d99 nodeName:}" failed. No retries permitted until 2026-09-06 12:54:39.172509913 +0000 UTC m=+488.914881049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-webhook-cert") pod "maas-controller-74d564bf49-s94c2" (UID: "0e9661e6-83b5-4cc7-b0e5-55de73661d99") : secret "maas-controller-webhook-cert" not found Sep 06 12:54:38.680497 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.680470 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fbnk\" (UniqueName: \"kubernetes.io/projected/0e9661e6-83b5-4cc7-b0e5-55de73661d99-kube-api-access-8fbnk\") pod \"maas-controller-74d564bf49-s94c2\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:38.773736 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.773656 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a128b654-e51a-42bf-94e9-6e9bb97ed624-webhook-cert\") pod \"maas-controller-777d65df8-f4lkl\" (UID: \"a128b654-e51a-42bf-94e9-6e9bb97ed624\") " pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:38.773736 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.773719 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjvd4\" (UniqueName: \"kubernetes.io/projected/a128b654-e51a-42bf-94e9-6e9bb97ed624-kube-api-access-cjvd4\") pod \"maas-controller-777d65df8-f4lkl\" (UID: \"a128b654-e51a-42bf-94e9-6e9bb97ed624\") " pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:38.773957 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.773859 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a128b654-e51a-42bf-94e9-6e9bb97ed624-metrics-certs\") pod \"maas-controller-777d65df8-f4lkl\" (UID: \"a128b654-e51a-42bf-94e9-6e9bb97ed624\") " pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:38.874348 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.874314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a128b654-e51a-42bf-94e9-6e9bb97ed624-webhook-cert\") pod \"maas-controller-777d65df8-f4lkl\" (UID: \"a128b654-e51a-42bf-94e9-6e9bb97ed624\") " pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:38.874539 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.874384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjvd4\" (UniqueName: \"kubernetes.io/projected/a128b654-e51a-42bf-94e9-6e9bb97ed624-kube-api-access-cjvd4\") pod \"maas-controller-777d65df8-f4lkl\" (UID: \"a128b654-e51a-42bf-94e9-6e9bb97ed624\") " pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:38.874539 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.874438 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a128b654-e51a-42bf-94e9-6e9bb97ed624-metrics-certs\") pod \"maas-controller-777d65df8-f4lkl\" (UID: \"a128b654-e51a-42bf-94e9-6e9bb97ed624\") " pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:38.874539 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:54:38.874522 2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 06 12:54:38.874705 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:54:38.874575 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a128b654-e51a-42bf-94e9-6e9bb97ed624-metrics-certs podName:a128b654-e51a-42bf-94e9-6e9bb97ed624 nodeName:}" failed. No retries permitted until 2026-09-06 12:54:39.37456153 +0000 UTC m=+489.116932662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a128b654-e51a-42bf-94e9-6e9bb97ed624-metrics-certs") pod "maas-controller-777d65df8-f4lkl" (UID: "a128b654-e51a-42bf-94e9-6e9bb97ed624") : secret "maas-controller-metrics-tls" not found Sep 06 12:54:38.876922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.876889 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a128b654-e51a-42bf-94e9-6e9bb97ed624-webhook-cert\") pod \"maas-controller-777d65df8-f4lkl\" (UID: \"a128b654-e51a-42bf-94e9-6e9bb97ed624\") " pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:38.882208 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:38.882182 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjvd4\" (UniqueName: \"kubernetes.io/projected/a128b654-e51a-42bf-94e9-6e9bb97ed624-kube-api-access-cjvd4\") pod \"maas-controller-777d65df8-f4lkl\" (UID: \"a128b654-e51a-42bf-94e9-6e9bb97ed624\") " pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:39.177120 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:39.177083 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-metrics-certs\") pod \"maas-controller-74d564bf49-s94c2\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:39.177275 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:39.177169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-webhook-cert\") pod \"maas-controller-74d564bf49-s94c2\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:39.177275 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:54:39.177239 2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 06 12:54:39.177366 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:54:39.177322 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-metrics-certs podName:0e9661e6-83b5-4cc7-b0e5-55de73661d99 nodeName:}" failed. No retries permitted until 2026-09-06 12:54:40.17730343 +0000 UTC m=+489.919674577 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-metrics-certs") pod "maas-controller-74d564bf49-s94c2" (UID: "0e9661e6-83b5-4cc7-b0e5-55de73661d99") : secret "maas-controller-metrics-tls" not found Sep 06 12:54:39.179706 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:39.179677 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-webhook-cert\") pod \"maas-controller-74d564bf49-s94c2\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:39.378575 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:39.378540 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a128b654-e51a-42bf-94e9-6e9bb97ed624-metrics-certs\") pod \"maas-controller-777d65df8-f4lkl\" (UID: \"a128b654-e51a-42bf-94e9-6e9bb97ed624\") " pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:39.380873 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:39.380842 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a128b654-e51a-42bf-94e9-6e9bb97ed624-metrics-certs\") pod \"maas-controller-777d65df8-f4lkl\" (UID: \"a128b654-e51a-42bf-94e9-6e9bb97ed624\") " pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:39.548890 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:39.548804 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:39.817949 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:39.817922 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-777d65df8-f4lkl"] Sep 06 12:54:39.819494 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:54:39.819470 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda128b654_e51a_42bf_94e9_6e9bb97ed624.slice/crio-251adee8fbc9837d09aa9f2de88dfac52010198ff0044bb21a65544efb01abd5 WatchSource:0}: Error finding container 251adee8fbc9837d09aa9f2de88dfac52010198ff0044bb21a65544efb01abd5: Status 404 returned error can't find the container with id 251adee8fbc9837d09aa9f2de88dfac52010198ff0044bb21a65544efb01abd5 Sep 06 12:54:40.017967 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:40.017933 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-777d65df8-f4lkl" event={"ID":"a128b654-e51a-42bf-94e9-6e9bb97ed624","Type":"ContainerStarted","Data":"251adee8fbc9837d09aa9f2de88dfac52010198ff0044bb21a65544efb01abd5"} Sep 06 12:54:40.185321 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:40.185216 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-metrics-certs\") pod \"maas-controller-74d564bf49-s94c2\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:40.187696 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:40.187676 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-metrics-certs\") pod \"maas-controller-74d564bf49-s94c2\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:40.280007 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:40.279963 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:40.564974 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:40.564840 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74d564bf49-s94c2"] Sep 06 12:54:41.024471 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:41.024389 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d564bf49-s94c2" event={"ID":"0e9661e6-83b5-4cc7-b0e5-55de73661d99","Type":"ContainerStarted","Data":"eee563bd4617104472d54e8e6c67216c9b82051f0788fd7adffb8fa9bbfba4d5"} Sep 06 12:54:43.036027 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:43.035985 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-777d65df8-f4lkl" event={"ID":"a128b654-e51a-42bf-94e9-6e9bb97ed624","Type":"ContainerStarted","Data":"ddfa618a63a8302da67b79a8b8faf2be8de73d8bf406a3a1cdca732bec2b6509"} Sep 06 12:54:43.036508 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:43.036061 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:43.037457 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:43.037428 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d564bf49-s94c2" event={"ID":"0e9661e6-83b5-4cc7-b0e5-55de73661d99","Type":"ContainerStarted","Data":"81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed"} Sep 06 12:54:43.037578 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:43.037501 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:43.054716 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:43.054670 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-777d65df8-f4lkl" podStartSLOduration=2.512507781 podStartE2EDuration="5.054655619s" totalImagesPullingTime="2.542147838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:54:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:54:43.052936784 +0000 UTC m=+492.795307939" watchObservedRunningTime="2026-09-06 12:54:43.054655619 +0000 UTC m=+492.797026774" Sep 06 12:54:43.069222 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:43.069180 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74d564bf49-s94c2" podStartSLOduration=3.259670116 podStartE2EDuration="5.06916442s" totalImagesPullingTime="1.809494304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:54:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:54:43.068131685 +0000 UTC m=+492.810502841" watchObservedRunningTime="2026-09-06 12:54:43.06916442 +0000 UTC m=+492.811535574" Sep 06 12:54:54.046385 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.046353 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:54.046848 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.046426 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-777d65df8-f4lkl" Sep 06 12:54:54.103414 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.103381 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74d564bf49-s94c2"] Sep 06 12:54:54.103649 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.103624 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74d564bf49-s94c2" podUID="0e9661e6-83b5-4cc7-b0e5-55de73661d99" containerName="manager" containerID="cri-o://81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed" gracePeriod=30 Sep 06 12:54:54.341016 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.340989 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:54.507729 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.507690 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fbnk\" (UniqueName: \"kubernetes.io/projected/0e9661e6-83b5-4cc7-b0e5-55de73661d99-kube-api-access-8fbnk\") pod \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " Sep 06 12:54:54.507922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.507755 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-metrics-certs\") pod \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " Sep 06 12:54:54.507922 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.507778 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-webhook-cert\") pod \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\" (UID: \"0e9661e6-83b5-4cc7-b0e5-55de73661d99\") " Sep 06 12:54:54.510059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.510029 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e9661e6-83b5-4cc7-b0e5-55de73661d99-kube-api-access-8fbnk" (OuterVolumeSpecName: "kube-api-access-8fbnk") pod "0e9661e6-83b5-4cc7-b0e5-55de73661d99" (UID: "0e9661e6-83b5-4cc7-b0e5-55de73661d99"). InnerVolumeSpecName "kube-api-access-8fbnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 12:54:54.510059 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.510045 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0e9661e6-83b5-4cc7-b0e5-55de73661d99" (UID: "0e9661e6-83b5-4cc7-b0e5-55de73661d99"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 12:54:54.510211 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.510066 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0e9661e6-83b5-4cc7-b0e5-55de73661d99" (UID: "0e9661e6-83b5-4cc7-b0e5-55de73661d99"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 12:54:54.608985 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.608901 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fbnk\" (UniqueName: \"kubernetes.io/projected/0e9661e6-83b5-4cc7-b0e5-55de73661d99-kube-api-access-8fbnk\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:54:54.608985 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.608933 2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-metrics-certs\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:54:54.608985 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:54.608943 2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e9661e6-83b5-4cc7-b0e5-55de73661d99-webhook-cert\") on node \"ip-10-0-141-13.ec2.internal\" DevicePath \"\"" Sep 06 12:54:55.094315 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:55.094282 2580 generic.go:356] "Generic (PLEG): container finished" podID="0e9661e6-83b5-4cc7-b0e5-55de73661d99" containerID="81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed" exitCode=0 Sep 06 12:54:55.094727 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:55.094325 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d564bf49-s94c2" Sep 06 12:54:55.094727 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:55.094345 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d564bf49-s94c2" event={"ID":"0e9661e6-83b5-4cc7-b0e5-55de73661d99","Type":"ContainerDied","Data":"81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed"} Sep 06 12:54:55.094727 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:55.094400 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d564bf49-s94c2" event={"ID":"0e9661e6-83b5-4cc7-b0e5-55de73661d99","Type":"ContainerDied","Data":"eee563bd4617104472d54e8e6c67216c9b82051f0788fd7adffb8fa9bbfba4d5"} Sep 06 12:54:55.094727 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:55.094422 2580 scope.go:117] "RemoveContainer" containerID="81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed" Sep 06 12:54:55.102758 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:55.102743 2580 scope.go:117] "RemoveContainer" containerID="81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed" Sep 06 12:54:55.103035 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:54:55.103014 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed\": container with ID starting with 81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed not found: ID does not exist" containerID="81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed" Sep 06 12:54:55.103104 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:55.103041 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed"} err="failed to get container status \"81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed\": rpc error: code = NotFound desc = could not find container \"81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed\": container with ID starting with 81d9e6a08034a28f0fba30ba82d3776e5678cccea865ab68d02e97a8263295ed not found: ID does not exist" Sep 06 12:54:55.109779 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:55.109739 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74d564bf49-s94c2"] Sep 06 12:54:55.111673 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:55.111652 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74d564bf49-s94c2"] Sep 06 12:54:56.735637 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:54:56.735607 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e9661e6-83b5-4cc7-b0e5-55de73661d99" path="/var/lib/kubelet/pods/0e9661e6-83b5-4cc7-b0e5-55de73661d99/volumes" Sep 06 12:55:16.819725 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.819687 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9"] Sep 06 12:55:16.820143 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.820018 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e9661e6-83b5-4cc7-b0e5-55de73661d99" containerName="manager" Sep 06 12:55:16.820143 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.820030 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e9661e6-83b5-4cc7-b0e5-55de73661d99" containerName="manager" Sep 06 12:55:16.820143 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.820099 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e9661e6-83b5-4cc7-b0e5-55de73661d99" containerName="manager" Sep 06 12:55:16.826702 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.826678 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:16.828881 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.828858 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 06 12:55:16.828998 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.828974 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6l8s7\"" Sep 06 12:55:16.832753 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.832722 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9"] Sep 06 12:55:16.883309 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.883272 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18ffbb6-5f3f-4715-be72-4195c934563d-openshift-service-ca\") pod \"maas-api-f57cbc8cb-p87c9\" (UID: \"b18ffbb6-5f3f-4715-be72-4195c934563d\") " pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:16.883510 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.883374 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b18ffbb6-5f3f-4715-be72-4195c934563d-maas-api-tls\") pod \"maas-api-f57cbc8cb-p87c9\" (UID: \"b18ffbb6-5f3f-4715-be72-4195c934563d\") " pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:16.883510 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.883402 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwzw4\" (UniqueName: \"kubernetes.io/projected/b18ffbb6-5f3f-4715-be72-4195c934563d-kube-api-access-xwzw4\") pod \"maas-api-f57cbc8cb-p87c9\" (UID: \"b18ffbb6-5f3f-4715-be72-4195c934563d\") " pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:16.984392 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.984351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b18ffbb6-5f3f-4715-be72-4195c934563d-maas-api-tls\") pod \"maas-api-f57cbc8cb-p87c9\" (UID: \"b18ffbb6-5f3f-4715-be72-4195c934563d\") " pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:16.984392 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.984395 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwzw4\" (UniqueName: \"kubernetes.io/projected/b18ffbb6-5f3f-4715-be72-4195c934563d-kube-api-access-xwzw4\") pod \"maas-api-f57cbc8cb-p87c9\" (UID: \"b18ffbb6-5f3f-4715-be72-4195c934563d\") " pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:16.984604 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.984429 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18ffbb6-5f3f-4715-be72-4195c934563d-openshift-service-ca\") pod \"maas-api-f57cbc8cb-p87c9\" (UID: \"b18ffbb6-5f3f-4715-be72-4195c934563d\") " pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:16.984604 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:55:16.984509 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Sep 06 12:55:16.984604 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:55:16.984597 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18ffbb6-5f3f-4715-be72-4195c934563d-maas-api-tls podName:b18ffbb6-5f3f-4715-be72-4195c934563d nodeName:}" failed. No retries permitted until 2026-09-06 12:55:17.484578439 +0000 UTC m=+527.226949574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b18ffbb6-5f3f-4715-be72-4195c934563d-maas-api-tls") pod "maas-api-f57cbc8cb-p87c9" (UID: "b18ffbb6-5f3f-4715-be72-4195c934563d") : secret "maas-api-serving-cert" not found Sep 06 12:55:16.985040 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.985022 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18ffbb6-5f3f-4715-be72-4195c934563d-openshift-service-ca\") pod \"maas-api-f57cbc8cb-p87c9\" (UID: \"b18ffbb6-5f3f-4715-be72-4195c934563d\") " pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:16.992794 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:16.992766 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwzw4\" (UniqueName: \"kubernetes.io/projected/b18ffbb6-5f3f-4715-be72-4195c934563d-kube-api-access-xwzw4\") pod \"maas-api-f57cbc8cb-p87c9\" (UID: \"b18ffbb6-5f3f-4715-be72-4195c934563d\") " pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:17.315370 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.315338 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz"] Sep 06 12:55:17.318756 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.318736 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" Sep 06 12:55:17.320942 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.320914 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 06 12:55:17.321075 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.320943 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rgr7g\"" Sep 06 12:55:17.329364 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.329330 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz"] Sep 06 12:55:17.388691 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.388654 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d6d8874-fa5c-433e-8720-4682f258089c-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-zhskz\" (UID: \"8d6d8874-fa5c-433e-8720-4682f258089c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" Sep 06 12:55:17.388864 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.388735 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lt5r\" (UniqueName: \"kubernetes.io/projected/8d6d8874-fa5c-433e-8720-4682f258089c-kube-api-access-8lt5r\") pod \"payload-pre-processing-59fbc47bf8-zhskz\" (UID: \"8d6d8874-fa5c-433e-8720-4682f258089c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" Sep 06 12:55:17.490079 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.490035 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b18ffbb6-5f3f-4715-be72-4195c934563d-maas-api-tls\") pod \"maas-api-f57cbc8cb-p87c9\" (UID: \"b18ffbb6-5f3f-4715-be72-4195c934563d\") " pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:17.490291 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.490109 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d6d8874-fa5c-433e-8720-4682f258089c-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-zhskz\" (UID: \"8d6d8874-fa5c-433e-8720-4682f258089c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" Sep 06 12:55:17.490291 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.490165 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lt5r\" (UniqueName: \"kubernetes.io/projected/8d6d8874-fa5c-433e-8720-4682f258089c-kube-api-access-8lt5r\") pod \"payload-pre-processing-59fbc47bf8-zhskz\" (UID: \"8d6d8874-fa5c-433e-8720-4682f258089c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" Sep 06 12:55:17.490881 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.490854 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d6d8874-fa5c-433e-8720-4682f258089c-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-zhskz\" (UID: \"8d6d8874-fa5c-433e-8720-4682f258089c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" Sep 06 12:55:17.492594 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.492575 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b18ffbb6-5f3f-4715-be72-4195c934563d-maas-api-tls\") pod \"maas-api-f57cbc8cb-p87c9\" (UID: \"b18ffbb6-5f3f-4715-be72-4195c934563d\") " pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:17.498285 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.498244 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lt5r\" (UniqueName: \"kubernetes.io/projected/8d6d8874-fa5c-433e-8720-4682f258089c-kube-api-access-8lt5r\") pod \"payload-pre-processing-59fbc47bf8-zhskz\" (UID: \"8d6d8874-fa5c-433e-8720-4682f258089c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" Sep 06 12:55:17.630365 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.630242 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" Sep 06 12:55:17.738972 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.738931 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:17.901396 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:17.901367 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz"] Sep 06 12:55:17.904236 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:55:17.904206 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d6d8874_fa5c_433e_8720_4682f258089c.slice/crio-352fa1da82dadb3ba21273951487e2e86166b3fd7b6138bc8345829393b21009 WatchSource:0}: Error finding container 352fa1da82dadb3ba21273951487e2e86166b3fd7b6138bc8345829393b21009: Status 404 returned error can't find the container with id 352fa1da82dadb3ba21273951487e2e86166b3fd7b6138bc8345829393b21009 Sep 06 12:55:18.013614 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:18.013556 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9"] Sep 06 12:55:18.018903 ip-10-0-141-13 kubenswrapper[2580]: W0906 12:55:18.018864 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18ffbb6_5f3f_4715_be72_4195c934563d.slice/crio-3db7c63597c0dee7907d0c38205c7575a5cb65fe9abae980375552685d32b36f WatchSource:0}: Error finding container 3db7c63597c0dee7907d0c38205c7575a5cb65fe9abae980375552685d32b36f: Status 404 returned error can't find the container with id 3db7c63597c0dee7907d0c38205c7575a5cb65fe9abae980375552685d32b36f Sep 06 12:55:18.202243 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:18.202151 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" event={"ID":"b18ffbb6-5f3f-4715-be72-4195c934563d","Type":"ContainerStarted","Data":"3db7c63597c0dee7907d0c38205c7575a5cb65fe9abae980375552685d32b36f"} Sep 06 12:55:18.203536 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:18.203510 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" event={"ID":"8d6d8874-fa5c-433e-8720-4682f258089c","Type":"ContainerStarted","Data":"352fa1da82dadb3ba21273951487e2e86166b3fd7b6138bc8345829393b21009"} Sep 06 12:55:22.229899 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:22.229811 2580 generic.go:356] "Generic (PLEG): container finished" podID="b18ffbb6-5f3f-4715-be72-4195c934563d" containerID="b279e760b6518f738645282fbb4751d1d92da9f2849043c160cd7027cf4ce3b0" exitCode=1 Sep 06 12:55:22.230367 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:22.229900 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" event={"ID":"b18ffbb6-5f3f-4715-be72-4195c934563d","Type":"ContainerDied","Data":"b279e760b6518f738645282fbb4751d1d92da9f2849043c160cd7027cf4ce3b0"} Sep 06 12:55:22.230367 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:22.230102 2580 scope.go:117] "RemoveContainer" containerID="b279e760b6518f738645282fbb4751d1d92da9f2849043c160cd7027cf4ce3b0" Sep 06 12:55:22.231532 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:22.231508 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" event={"ID":"8d6d8874-fa5c-433e-8720-4682f258089c","Type":"ContainerStarted","Data":"8934c6fb34655cf901527ae2bb2dde0f13cb31bfb3f3a86f30a7c47c15233c53"} Sep 06 12:55:22.231649 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:22.231577 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" Sep 06 12:55:22.740136 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:22.740093 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:23.239712 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:23.237767 2580 generic.go:356] "Generic (PLEG): container finished" podID="b18ffbb6-5f3f-4715-be72-4195c934563d" containerID="c8ef7d9e96d7e4cd0b4eef5e35b27b01d3150e9d394fbdf8ee22dbddc99fa35c" exitCode=1 Sep 06 12:55:23.239712 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:23.238773 2580 scope.go:117] "RemoveContainer" containerID="c8ef7d9e96d7e4cd0b4eef5e35b27b01d3150e9d394fbdf8ee22dbddc99fa35c" Sep 06 12:55:23.239712 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:55:23.238953 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:55:23.239712 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:23.239206 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" event={"ID":"b18ffbb6-5f3f-4715-be72-4195c934563d","Type":"ContainerDied","Data":"c8ef7d9e96d7e4cd0b4eef5e35b27b01d3150e9d394fbdf8ee22dbddc99fa35c"} Sep 06 12:55:23.239712 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:23.239287 2580 scope.go:117] "RemoveContainer" containerID="b279e760b6518f738645282fbb4751d1d92da9f2849043c160cd7027cf4ce3b0" Sep 06 12:55:23.255582 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:23.255527 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" podStartSLOduration=2.647281967 podStartE2EDuration="6.255486224s" totalImagesPullingTime="3.608204257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 12:55:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 12:55:22.269476785 +0000 UTC m=+532.011847943" watchObservedRunningTime="2026-09-06 12:55:23.255486224 +0000 UTC m=+532.997857380" Sep 06 12:55:24.245237 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:24.245202 2580 scope.go:117] "RemoveContainer" containerID="c8ef7d9e96d7e4cd0b4eef5e35b27b01d3150e9d394fbdf8ee22dbddc99fa35c" Sep 06 12:55:24.245664 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:55:24.245492 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:55:25.251062 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:25.251025 2580 scope.go:117] "RemoveContainer" containerID="c8ef7d9e96d7e4cd0b4eef5e35b27b01d3150e9d394fbdf8ee22dbddc99fa35c" Sep 06 12:55:25.251668 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:55:25.251193 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:55:27.261441 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:27.261404 2580 generic.go:356] "Generic (PLEG): container finished" podID="5178826f-8d7f-4be4-871b-528202e9198d" containerID="c1c76e7287ddf2df77f5d3ec5c163fbe9940abc9ba3014d3195dc3c50079addb" exitCode=0 Sep 06 12:55:27.261913 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:27.261476 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" event={"ID":"5178826f-8d7f-4be4-871b-528202e9198d","Type":"ContainerDied","Data":"c1c76e7287ddf2df77f5d3ec5c163fbe9940abc9ba3014d3195dc3c50079addb"} Sep 06 12:55:27.261913 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:27.261873 2580 scope.go:117] "RemoveContainer" containerID="c1c76e7287ddf2df77f5d3ec5c163fbe9940abc9ba3014d3195dc3c50079addb" Sep 06 12:55:27.262864 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:27.262839 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 06 12:55:27.262940 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:27.262921 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 06 12:55:27.262981 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:27.262962 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 06 12:55:27.740058 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:27.740025 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:27.740222 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:27.740087 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:27.740443 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:27.740430 2580 scope.go:117] "RemoveContainer" containerID="c8ef7d9e96d7e4cd0b4eef5e35b27b01d3150e9d394fbdf8ee22dbddc99fa35c" Sep 06 12:55:27.740615 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:55:27.740598 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:55:28.268295 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:28.268244 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" event={"ID":"5178826f-8d7f-4be4-871b-528202e9198d","Type":"ContainerStarted","Data":"196b09da391e830fa807fe0270d2f7a90b61664a0f1ad0497bb591cbe72cf067"} Sep 06 12:55:29.268223 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:29.268179 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:55:29.269540 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:29.269510 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6gs8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Sep 06 12:55:29.269873 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:29.269567 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" podUID="5178826f-8d7f-4be4-871b-528202e9198d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Sep 06 12:55:30.268337 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:30.268299 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6gs8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Sep 06 12:55:30.268525 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:30.268356 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" podUID="5178826f-8d7f-4be4-871b-528202e9198d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Sep 06 12:55:31.268789 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:31.268749 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6gs8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Sep 06 12:55:31.269181 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:31.268810 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" podUID="5178826f-8d7f-4be4-871b-528202e9198d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Sep 06 12:55:31.829225 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:31.829190 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:55:32.268664 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:32.268558 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6gs8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Sep 06 12:55:32.268664 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:32.268635 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" podUID="5178826f-8d7f-4be4-871b-528202e9198d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Sep 06 12:55:33.241466 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:33.241431 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zhskz" Sep 06 12:55:33.268417 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:33.268381 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6gs8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Sep 06 12:55:33.268580 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:33.268441 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" podUID="5178826f-8d7f-4be4-871b-528202e9198d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Sep 06 12:55:34.272732 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:34.272699 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:55:34.273683 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:34.273665 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6gs8m" Sep 06 12:55:38.730628 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:38.730593 2580 scope.go:117] "RemoveContainer" containerID="c8ef7d9e96d7e4cd0b4eef5e35b27b01d3150e9d394fbdf8ee22dbddc99fa35c" Sep 06 12:55:40.324162 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:40.324130 2580 generic.go:356] "Generic (PLEG): container finished" podID="b18ffbb6-5f3f-4715-be72-4195c934563d" containerID="710977b6b632f87f6c1f48c678c9330d776549054fb719ecaf02d255be3b9603" exitCode=1 Sep 06 12:55:40.324623 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:40.324201 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" event={"ID":"b18ffbb6-5f3f-4715-be72-4195c934563d","Type":"ContainerDied","Data":"710977b6b632f87f6c1f48c678c9330d776549054fb719ecaf02d255be3b9603"} Sep 06 12:55:40.324623 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:40.324278 2580 scope.go:117] "RemoveContainer" containerID="c8ef7d9e96d7e4cd0b4eef5e35b27b01d3150e9d394fbdf8ee22dbddc99fa35c" Sep 06 12:55:40.324623 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:40.324582 2580 scope.go:117] "RemoveContainer" containerID="710977b6b632f87f6c1f48c678c9330d776549054fb719ecaf02d255be3b9603" Sep 06 12:55:40.324812 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:55:40.324793 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:55:42.739568 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:42.739541 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:42.739978 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:42.739872 2580 scope.go:117] "RemoveContainer" containerID="710977b6b632f87f6c1f48c678c9330d776549054fb719ecaf02d255be3b9603" Sep 06 12:55:42.740054 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:55:42.740039 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:55:47.739161 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:47.739127 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:55:47.739656 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:55:47.739577 2580 scope.go:117] "RemoveContainer" containerID="710977b6b632f87f6c1f48c678c9330d776549054fb719ecaf02d255be3b9603" Sep 06 12:55:47.739805 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:55:47.739781 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:56:02.736428 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:02.736397 2580 scope.go:117] "RemoveContainer" containerID="710977b6b632f87f6c1f48c678c9330d776549054fb719ecaf02d255be3b9603" Sep 06 12:56:03.430127 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:03.430089 2580 generic.go:356] "Generic (PLEG): container finished" podID="b18ffbb6-5f3f-4715-be72-4195c934563d" containerID="c92a9757225a6e699eb571ba52ad05016f59958d76c70b226bab9ca82743d91c" exitCode=1 Sep 06 12:56:03.430330 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:03.430132 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" event={"ID":"b18ffbb6-5f3f-4715-be72-4195c934563d","Type":"ContainerDied","Data":"c92a9757225a6e699eb571ba52ad05016f59958d76c70b226bab9ca82743d91c"} Sep 06 12:56:03.430330 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:03.430190 2580 scope.go:117] "RemoveContainer" containerID="710977b6b632f87f6c1f48c678c9330d776549054fb719ecaf02d255be3b9603" Sep 06 12:56:03.430654 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:03.430633 2580 scope.go:117] "RemoveContainer" containerID="c92a9757225a6e699eb571ba52ad05016f59958d76c70b226bab9ca82743d91c" Sep 06 12:56:03.430876 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:56:03.430855 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:56:07.739135 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:07.739096 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:56:07.739541 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:07.739185 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:56:07.739593 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:07.739553 2580 scope.go:117] "RemoveContainer" containerID="c92a9757225a6e699eb571ba52ad05016f59958d76c70b226bab9ca82743d91c" Sep 06 12:56:07.739761 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:56:07.739741 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:56:22.730775 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:22.730737 2580 scope.go:117] "RemoveContainer" containerID="c92a9757225a6e699eb571ba52ad05016f59958d76c70b226bab9ca82743d91c" Sep 06 12:56:22.731170 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:56:22.730969 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:56:30.710141 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:30.710108 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 12:56:30.711647 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:30.711620 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 12:56:33.730136 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:33.730104 2580 scope.go:117] "RemoveContainer" containerID="c92a9757225a6e699eb571ba52ad05016f59958d76c70b226bab9ca82743d91c" Sep 06 12:56:33.730625 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:56:33.730284 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:56:45.730668 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:45.730635 2580 scope.go:117] "RemoveContainer" containerID="c92a9757225a6e699eb571ba52ad05016f59958d76c70b226bab9ca82743d91c" Sep 06 12:56:46.625209 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:46.625129 2580 generic.go:356] "Generic (PLEG): container finished" podID="b18ffbb6-5f3f-4715-be72-4195c934563d" containerID="37cc9e3af4ca3c3710ba1572ec2c347df4ab69e4c41c571403daaa1138d26767" exitCode=1 Sep 06 12:56:46.625209 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:46.625196 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" event={"ID":"b18ffbb6-5f3f-4715-be72-4195c934563d","Type":"ContainerDied","Data":"37cc9e3af4ca3c3710ba1572ec2c347df4ab69e4c41c571403daaa1138d26767"} Sep 06 12:56:46.625417 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:46.625298 2580 scope.go:117] "RemoveContainer" containerID="c92a9757225a6e699eb571ba52ad05016f59958d76c70b226bab9ca82743d91c" Sep 06 12:56:46.625585 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:46.625568 2580 scope.go:117] "RemoveContainer" containerID="37cc9e3af4ca3c3710ba1572ec2c347df4ab69e4c41c571403daaa1138d26767" Sep 06 12:56:46.625792 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:56:46.625771 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:56:47.739757 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:47.739726 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:56:47.740223 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:47.739782 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:56:47.740223 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:47.740117 2580 scope.go:117] "RemoveContainer" containerID="37cc9e3af4ca3c3710ba1572ec2c347df4ab69e4c41c571403daaa1138d26767" Sep 06 12:56:47.740387 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:56:47.740364 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:56:58.730380 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:56:58.730306 2580 scope.go:117] "RemoveContainer" containerID="37cc9e3af4ca3c3710ba1572ec2c347df4ab69e4c41c571403daaa1138d26767" Sep 06 12:56:58.730737 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:56:58.730469 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:57:09.730728 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:57:09.730696 2580 scope.go:117] "RemoveContainer" containerID="37cc9e3af4ca3c3710ba1572ec2c347df4ab69e4c41c571403daaa1138d26767" Sep 06 12:57:09.731194 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:57:09.730877 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:57:21.730306 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:57:21.730249 2580 scope.go:117] "RemoveContainer" containerID="37cc9e3af4ca3c3710ba1572ec2c347df4ab69e4c41c571403daaa1138d26767" Sep 06 12:57:21.730689 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:57:21.730456 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:57:36.730232 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:57:36.730194 2580 scope.go:117] "RemoveContainer" containerID="37cc9e3af4ca3c3710ba1572ec2c347df4ab69e4c41c571403daaa1138d26767" Sep 06 12:57:36.730749 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:57:36.730429 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:57:47.730567 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:57:47.730535 2580 scope.go:117] "RemoveContainer" containerID="37cc9e3af4ca3c3710ba1572ec2c347df4ab69e4c41c571403daaa1138d26767" Sep 06 12:57:47.731133 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:57:47.730738 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:58:00.732847 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:00.732819 2580 scope.go:117] "RemoveContainer" containerID="37cc9e3af4ca3c3710ba1572ec2c347df4ab69e4c41c571403daaa1138d26767" Sep 06 12:58:00.733294 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:58:00.733054 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:58:13.730448 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:13.730411 2580 scope.go:117] "RemoveContainer" containerID="37cc9e3af4ca3c3710ba1572ec2c347df4ab69e4c41c571403daaa1138d26767" Sep 06 12:58:13.730819 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:13.730715 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 12:58:15.026251 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:15.026218 2580 generic.go:356] "Generic (PLEG): container finished" podID="b18ffbb6-5f3f-4715-be72-4195c934563d" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" exitCode=1 Sep 06 12:58:15.026754 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:15.026281 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" event={"ID":"b18ffbb6-5f3f-4715-be72-4195c934563d","Type":"ContainerDied","Data":"bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca"} Sep 06 12:58:15.026754 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:15.026334 2580 scope.go:117] "RemoveContainer" containerID="37cc9e3af4ca3c3710ba1572ec2c347df4ab69e4c41c571403daaa1138d26767" Sep 06 12:58:15.026754 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:15.026675 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 12:58:15.026948 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:58:15.026927 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:58:17.739445 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:17.739394 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:58:17.739849 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:17.739473 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 12:58:17.739849 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:17.739792 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 12:58:17.739973 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:58:17.739954 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:58:28.730420 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:28.730326 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 12:58:28.730965 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:58:28.730556 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:58:42.730510 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:42.730474 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 12:58:42.730926 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:58:42.730705 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:58:56.736257 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:58:56.736219 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 12:58:56.736763 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:58:56.736480 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:59:10.735639 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:59:10.735610 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 12:59:10.736093 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:59:10.735784 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:59:22.730114 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:59:22.730077 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 12:59:22.730568 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:59:22.730351 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:59:36.730490 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:59:36.730457 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 12:59:36.730884 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:59:36.730628 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 12:59:48.731099 ip-10-0-141-13 kubenswrapper[2580]: I0906 12:59:48.731064 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 12:59:48.731608 ip-10-0-141-13 kubenswrapper[2580]: E0906 12:59:48.731257 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:00:00.146209 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:00.146127 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55"] Sep 06 13:00:00.149477 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:00.149458 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" Sep 06 13:00:00.151625 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:00.151604 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9xdh\"" Sep 06 13:00:00.159001 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:00.158981 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" Sep 06 13:00:00.165030 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:00.164999 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55"] Sep 06 13:00:00.434484 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:00.434463 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55"] Sep 06 13:00:00.436909 ip-10-0-141-13 kubenswrapper[2580]: W0906 13:00:00.436888 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c0c92cc_335d_48b2_aa58_7544ca6eaeff.slice/crio-7b3efa2023d4602951120f64c61e595ef749cb1537745a61f2830600b0a7f06f WatchSource:0}: Error finding container 7b3efa2023d4602951120f64c61e595ef749cb1537745a61f2830600b0a7f06f: Status 404 returned error can't find the container with id 7b3efa2023d4602951120f64c61e595ef749cb1537745a61f2830600b0a7f06f Sep 06 13:00:00.497631 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:00.497598 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" event={"ID":"0c0c92cc-335d-48b2-aa58-7544ca6eaeff","Type":"ContainerStarted","Data":"7b3efa2023d4602951120f64c61e595ef749cb1537745a61f2830600b0a7f06f"} Sep 06 13:00:03.513251 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:03.513166 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" event={"ID":"0c0c92cc-335d-48b2-aa58-7544ca6eaeff","Type":"ContainerStarted","Data":"6ae3af87a655df35b827c8a785061c0cc18c74e27d7cbfccaf0fe27197a3c915"} Sep 06 13:00:03.527517 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:03.527466 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" podStartSLOduration=1.055256519 podStartE2EDuration="3.527451383s" totalImagesPullingTime="2.472194864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 13:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 13:00:03.525575764 +0000 UTC m=+813.267946923" watchObservedRunningTime="2026-09-06 13:00:03.527451383 +0000 UTC m=+813.269822537" Sep 06 13:00:03.730158 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:03.730118 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 13:00:03.730397 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:00:03.730359 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:00:04.519076 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:04.519040 2580 generic.go:356] "Generic (PLEG): container finished" podID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" containerID="6ae3af87a655df35b827c8a785061c0cc18c74e27d7cbfccaf0fe27197a3c915" exitCode=7 Sep 06 13:00:04.519472 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:04.519126 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" event={"ID":"0c0c92cc-335d-48b2-aa58-7544ca6eaeff","Type":"ContainerDied","Data":"6ae3af87a655df35b827c8a785061c0cc18c74e27d7cbfccaf0fe27197a3c915"} Sep 06 13:00:04.519472 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:04.519281 2580 scope.go:117] "RemoveContainer" containerID="6ae3af87a655df35b827c8a785061c0cc18c74e27d7cbfccaf0fe27197a3c915" Sep 06 13:00:05.525677 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:05.525589 2580 generic.go:356] "Generic (PLEG): container finished" podID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" containerID="620f608a40de3c087665d98d187aa947e04c83103532b98a1e7b63196f717604" exitCode=7 Sep 06 13:00:05.526063 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:05.525675 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" event={"ID":"0c0c92cc-335d-48b2-aa58-7544ca6eaeff","Type":"ContainerDied","Data":"620f608a40de3c087665d98d187aa947e04c83103532b98a1e7b63196f717604"} Sep 06 13:00:05.526063 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:05.525737 2580 scope.go:117] "RemoveContainer" containerID="6ae3af87a655df35b827c8a785061c0cc18c74e27d7cbfccaf0fe27197a3c915" Sep 06 13:00:05.526063 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:05.525926 2580 scope.go:117] "RemoveContainer" containerID="620f608a40de3c087665d98d187aa947e04c83103532b98a1e7b63196f717604" Sep 06 13:00:05.526210 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:00:05.526166 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29811660-jxg55_odh-ai-gateway-infra(0c0c92cc-335d-48b2-aa58-7544ca6eaeff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" Sep 06 13:00:06.531673 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:06.531648 2580 scope.go:117] "RemoveContainer" containerID="620f608a40de3c087665d98d187aa947e04c83103532b98a1e7b63196f717604" Sep 06 13:00:06.532047 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:00:06.531850 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29811660-jxg55_odh-ai-gateway-infra(0c0c92cc-335d-48b2-aa58-7544ca6eaeff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" Sep 06 13:00:18.730869 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:18.730828 2580 scope.go:117] "RemoveContainer" containerID="620f608a40de3c087665d98d187aa947e04c83103532b98a1e7b63196f717604" Sep 06 13:00:18.731374 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:18.731001 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 13:00:18.731374 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:00:18.731214 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:00:19.592067 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:19.592030 2580 generic.go:356] "Generic (PLEG): container finished" podID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" containerID="31bfd61c56629980121420b762197764cc16001b9090191db2a740f8f3a4208a" exitCode=7 Sep 06 13:00:19.592249 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:19.592107 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" event={"ID":"0c0c92cc-335d-48b2-aa58-7544ca6eaeff","Type":"ContainerDied","Data":"31bfd61c56629980121420b762197764cc16001b9090191db2a740f8f3a4208a"} Sep 06 13:00:19.592249 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:19.592168 2580 scope.go:117] "RemoveContainer" containerID="620f608a40de3c087665d98d187aa947e04c83103532b98a1e7b63196f717604" Sep 06 13:00:19.592436 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:19.592421 2580 scope.go:117] "RemoveContainer" containerID="31bfd61c56629980121420b762197764cc16001b9090191db2a740f8f3a4208a" Sep 06 13:00:19.592674 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:00:19.592656 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29811660-jxg55_odh-ai-gateway-infra(0c0c92cc-335d-48b2-aa58-7544ca6eaeff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" Sep 06 13:00:20.616429 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:20.616389 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55"] Sep 06 13:00:20.748553 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:20.748519 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" Sep 06 13:00:21.604653 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:21.604617 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" event={"ID":"0c0c92cc-335d-48b2-aa58-7544ca6eaeff","Type":"ContainerDied","Data":"7b3efa2023d4602951120f64c61e595ef749cb1537745a61f2830600b0a7f06f"} Sep 06 13:00:21.604653 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:21.604645 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55" Sep 06 13:00:21.604918 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:21.604724 2580 scope.go:117] "RemoveContainer" containerID="31bfd61c56629980121420b762197764cc16001b9090191db2a740f8f3a4208a" Sep 06 13:00:21.626764 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:21.626732 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55"] Sep 06 13:00:21.632180 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:21.632152 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811660-jxg55"] Sep 06 13:00:22.735835 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:22.735799 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" path="/var/lib/kubelet/pods/0c0c92cc-335d-48b2-aa58-7544ca6eaeff/volumes" Sep 06 13:00:30.732537 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:30.732505 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 13:00:30.735062 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:00:30.732720 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:00:44.730518 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:44.730484 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 13:00:44.731007 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:00:44.730643 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:00:57.730401 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:57.730372 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 13:00:58.779771 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:58.779682 2580 generic.go:356] "Generic (PLEG): container finished" podID="b18ffbb6-5f3f-4715-be72-4195c934563d" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" exitCode=1 Sep 06 13:00:58.779771 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:58.779745 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" event={"ID":"b18ffbb6-5f3f-4715-be72-4195c934563d","Type":"ContainerDied","Data":"2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e"} Sep 06 13:00:58.780300 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:58.779792 2580 scope.go:117] "RemoveContainer" containerID="bdc99d802004d26c33fac94a43c8ff236b4d616a291a5df118336a8f9e09e2ca" Sep 06 13:00:58.780300 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:00:58.780147 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:00:58.780397 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:00:58.780377 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:01:02.739469 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:01:02.739433 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 13:01:02.739868 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:01:02.739851 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:01:02.740061 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:01:02.740039 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:01:07.739383 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:01:07.739342 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 13:01:07.739807 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:01:07.739791 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:01:07.739980 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:01:07.739960 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:01:21.730905 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:01:21.730862 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:01:21.731504 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:01:21.731111 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:01:30.739916 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:01:30.739834 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 13:01:30.740330 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:01:30.740024 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 13:01:32.730445 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:01:32.730410 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:01:32.730920 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:01:32.730573 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:01:46.730457 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:01:46.730417 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:01:46.730946 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:01:46.730605 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:02:00.732816 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:02:00.732779 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:02:00.733309 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:02:00.732978 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:02:14.730731 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:02:14.730693 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:02:14.731201 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:02:14.730951 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:02:27.730630 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:02:27.730599 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:02:27.731015 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:02:27.730798 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:02:38.730742 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:02:38.730704 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:02:38.731354 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:02:38.730967 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:02:50.732294 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:02:50.732248 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:02:50.732677 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:02:50.732465 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:03:05.730701 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:03:05.730660 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:03:05.731155 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:03:05.730900 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:03:18.731370 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:03:18.731335 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:03:18.732428 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:03:18.732387 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:03:32.730509 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:03:32.730477 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:03:32.730887 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:03:32.730647 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:03:47.730748 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:03:47.730712 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:03:47.731173 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:03:47.730893 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:04:00.732691 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:04:00.732660 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:04:00.733253 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:04:00.732908 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:04:11.730523 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:04:11.730494 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:04:11.730898 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:04:11.730659 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:04:22.730030 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:04:22.729996 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:04:22.730439 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:04:22.730181 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:04:33.730113 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:04:33.730041 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:04:33.730590 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:04:33.730226 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:04:45.731006 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:04:45.730974 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:04:45.731419 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:04:45.731122 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:04:59.730660 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:04:59.730625 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:04:59.731120 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:04:59.730868 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:05:14.730763 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:05:14.730732 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:05:14.731238 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:05:14.730926 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:05:28.730529 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:05:28.730495 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:05:28.730970 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:05:28.730731 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:05:42.730335 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:05:42.730301 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:05:42.734194 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:05:42.730497 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:05:57.731143 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:05:57.731062 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:05:57.731612 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:05:57.731227 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:06:11.730741 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:11.730708 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:06:11.731225 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:11.731106 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 13:06:13.205946 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:13.205911 2580 generic.go:356] "Generic (PLEG): container finished" podID="b18ffbb6-5f3f-4715-be72-4195c934563d" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" exitCode=1 Sep 06 13:06:13.206371 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:13.205967 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" event={"ID":"b18ffbb6-5f3f-4715-be72-4195c934563d","Type":"ContainerDied","Data":"6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb"} Sep 06 13:06:13.206371 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:13.206013 2580 scope.go:117] "RemoveContainer" containerID="2cc00a32ff613971764d300675d004e156bdd5c3f84b58eed345427087c19e3e" Sep 06 13:06:13.206371 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:13.206324 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:06:13.206552 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:06:13.206511 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:06:17.739543 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:17.739512 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 13:06:17.739899 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:17.739567 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 13:06:17.739899 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:17.739826 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:06:17.740005 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:06:17.739986 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:06:30.768614 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:30.768579 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 13:06:30.770712 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:30.770693 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 13:06:32.730533 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:32.730505 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:06:32.730906 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:06:32.730670 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:06:46.731229 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:46.731191 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:06:46.731638 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:06:46.731414 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:06:58.730930 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:06:58.730900 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:06:58.731352 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:06:58.731079 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:07:11.730718 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:07:11.730686 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:07:11.731186 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:07:11.730863 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:07:24.730521 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:07:24.730439 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:07:24.731065 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:07:24.730619 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:07:35.729892 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:07:35.729860 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:07:35.730304 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:07:35.730034 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:07:46.736684 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:07:46.736652 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:07:46.737078 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:07:46.736875 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:07:57.730910 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:07:57.730877 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:07:57.731643 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:07:57.731060 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:08:12.730883 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:08:12.730852 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:08:12.731255 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:08:12.731038 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:08:26.730542 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:08:26.730505 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:08:26.731045 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:08:26.730680 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:08:38.730291 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:08:38.730232 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:08:38.730816 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:08:38.730541 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:08:52.736077 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:08:52.736029 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:08:52.736556 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:08:52.736234 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:09:07.730188 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:09:07.730157 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:09:07.730620 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:09:07.730368 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:09:20.732137 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:09:20.732106 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:09:20.732517 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:09:20.732294 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:09:33.730227 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:09:33.730195 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:09:33.730712 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:09:33.730412 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:09:44.730475 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:09:44.730444 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:09:44.730978 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:09:44.730615 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:09:56.736166 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:09:56.736138 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:09:56.736563 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:09:56.736338 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:10:10.732691 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:10:10.732653 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:10:10.733135 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:10:10.732842 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:10:22.736165 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:10:22.736131 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:10:22.736685 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:10:22.736351 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:10:33.729976 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:10:33.729889 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:10:33.730460 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:10:33.730165 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:10:45.730278 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:10:45.730242 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:10:45.732775 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:10:45.730481 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:10:59.730782 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:10:59.730749 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:10:59.731285 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:10:59.730983 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:11:11.729959 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:11.729925 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:11:11.730366 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:11:11.730101 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:11:22.730154 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:22.730117 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:11:22.730630 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:22.730456 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 13:11:23.603539 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:23.603505 2580 generic.go:356] "Generic (PLEG): container finished" podID="b18ffbb6-5f3f-4715-be72-4195c934563d" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" exitCode=1 Sep 06 13:11:23.603709 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:23.603577 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" event={"ID":"b18ffbb6-5f3f-4715-be72-4195c934563d","Type":"ContainerDied","Data":"89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9"} Sep 06 13:11:23.603709 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:23.603637 2580 scope.go:117] "RemoveContainer" containerID="6011461a12cc47120ea5889867e2a36ff031983d0bf09664951ffdd97399d0bb" Sep 06 13:11:23.603968 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:23.603946 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:11:23.604181 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:11:23.604162 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:11:27.739706 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:27.739676 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 13:11:27.740212 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:27.739771 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 13:11:27.740212 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:27.740133 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:11:27.740389 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:11:27.740367 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:11:28.626375 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:28.626348 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:11:28.626556 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:11:28.626534 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:11:30.795118 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:30.795088 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 13:11:30.798905 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:30.798881 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 13:11:41.730509 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:41.730474 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:11:41.730917 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:11:41.730697 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:11:55.730927 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:11:55.730845 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:11:55.731396 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:11:55.731037 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:12:09.730194 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:12:09.730156 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:12:09.730626 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:12:09.730393 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:12:22.736033 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:12:22.736003 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:12:22.736442 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:12:22.736172 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:12:34.730477 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:12:34.730445 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:12:34.730919 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:12:34.730669 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:12:45.730371 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:12:45.730341 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:12:45.730842 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:12:45.730501 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:12:59.730164 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:12:59.730126 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:12:59.730669 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:12:59.730388 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:13:13.730527 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:13:13.730495 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:13:13.730996 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:13:13.730643 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:13:25.729912 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:13:25.729831 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:13:25.730409 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:13:25.729996 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:13:39.730656 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:13:39.730625 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:13:39.731053 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:13:39.730808 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:13:50.734156 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:13:50.734129 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:13:50.734538 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:13:50.734321 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:14:04.736184 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:14:04.736158 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:14:04.736670 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:14:04.736335 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:14:16.735563 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:14:16.735532 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:14:16.735932 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:14:16.735708 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:14:29.730171 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:14:29.730135 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:14:29.730647 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:14:29.730383 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:14:44.730303 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:14:44.730247 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:14:44.730761 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:14:44.730439 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:14:57.730627 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:14:57.730549 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:14:57.731055 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:14:57.730768 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:15:00.129663 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.129628 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4"] Sep 06 13:15:00.130030 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.129958 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" containerName="cleanup" Sep 06 13:15:00.130030 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.129971 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" containerName="cleanup" Sep 06 13:15:00.130030 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.129985 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" containerName="cleanup" Sep 06 13:15:00.130030 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.129990 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" containerName="cleanup" Sep 06 13:15:00.130162 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.130049 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" containerName="cleanup" Sep 06 13:15:00.130162 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.130060 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" containerName="cleanup" Sep 06 13:15:00.133103 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.133087 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4" Sep 06 13:15:00.135242 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.135223 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9xdh\"" Sep 06 13:15:00.142285 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.142249 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4" Sep 06 13:15:00.144512 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.144489 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4"] Sep 06 13:15:00.411103 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.411044 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4"] Sep 06 13:15:00.413518 ip-10-0-141-13 kubenswrapper[2580]: W0906 13:15:00.413492 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57c0cce4_30b0_4904_af59_343ba6ea430a.slice/crio-617e22b7132c6673286ce4e71d7ee6726aedc2e2250c2c70d60c8b25d65b3d37 WatchSource:0}: Error finding container 617e22b7132c6673286ce4e71d7ee6726aedc2e2250c2c70d60c8b25d65b3d37: Status 404 returned error can't find the container with id 617e22b7132c6673286ce4e71d7ee6726aedc2e2250c2c70d60c8b25d65b3d37 Sep 06 13:15:00.560026 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:00.559995 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4" event={"ID":"57c0cce4-30b0-4904-af59-343ba6ea430a","Type":"ContainerStarted","Data":"617e22b7132c6673286ce4e71d7ee6726aedc2e2250c2c70d60c8b25d65b3d37"} Sep 06 13:15:01.566174 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:01.566142 2580 generic.go:356] "Generic (PLEG): container finished" podID="57c0cce4-30b0-4904-af59-343ba6ea430a" containerID="38fe02faf6ba390cca43a58b5286ee2164a1d09e891fa20b874cfae1d3cbeac9" exitCode=7 Sep 06 13:15:01.566640 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:01.566227 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4" event={"ID":"57c0cce4-30b0-4904-af59-343ba6ea430a","Type":"ContainerDied","Data":"38fe02faf6ba390cca43a58b5286ee2164a1d09e891fa20b874cfae1d3cbeac9"} Sep 06 13:15:01.566640 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:01.566375 2580 scope.go:117] "RemoveContainer" containerID="38fe02faf6ba390cca43a58b5286ee2164a1d09e891fa20b874cfae1d3cbeac9" Sep 06 13:15:02.573012 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:02.572974 2580 generic.go:356] "Generic (PLEG): container finished" podID="57c0cce4-30b0-4904-af59-343ba6ea430a" containerID="ca943d738d732a1a45e9713ac7e0fe946920a651dc8a36d6a81f9a98cb4943ce" exitCode=7 Sep 06 13:15:02.573439 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:02.573019 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4" event={"ID":"57c0cce4-30b0-4904-af59-343ba6ea430a","Type":"ContainerDied","Data":"ca943d738d732a1a45e9713ac7e0fe946920a651dc8a36d6a81f9a98cb4943ce"} Sep 06 13:15:02.573439 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:02.573073 2580 scope.go:117] "RemoveContainer" containerID="38fe02faf6ba390cca43a58b5286ee2164a1d09e891fa20b874cfae1d3cbeac9" Sep 06 13:15:02.573439 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:02.573198 2580 scope.go:117] "RemoveContainer" containerID="ca943d738d732a1a45e9713ac7e0fe946920a651dc8a36d6a81f9a98cb4943ce" Sep 06 13:15:02.573814 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:15:02.573440 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29811675-frnq4_odh-ai-gateway-infra(57c0cce4-30b0-4904-af59-343ba6ea430a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" Sep 06 13:15:03.578927 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:03.578899 2580 scope.go:117] "RemoveContainer" containerID="ca943d738d732a1a45e9713ac7e0fe946920a651dc8a36d6a81f9a98cb4943ce" Sep 06 13:15:03.579408 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:15:03.579102 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29811675-frnq4_odh-ai-gateway-infra(57c0cce4-30b0-4904-af59-343ba6ea430a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" Sep 06 13:15:06.356773 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.356738 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk"] Sep 06 13:15:06.357159 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.357145 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" containerName="cleanup" Sep 06 13:15:06.357203 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.357160 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" containerName="cleanup" Sep 06 13:15:06.357252 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.357241 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c0c92cc-335d-48b2-aa58-7544ca6eaeff" containerName="cleanup" Sep 06 13:15:06.361530 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.361508 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.363877 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.363695 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 06 13:15:06.363877 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.363743 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 06 13:15:06.364077 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.363925 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 06 13:15:06.364368 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.364350 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vsk2d\"" Sep 06 13:15:06.371611 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.371580 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk"] Sep 06 13:15:06.426847 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.426805 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dd827b-b347-46a4-b0b1-8c61ac63a401-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.427030 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.426866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dd827b-b347-46a4-b0b1-8c61ac63a401-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.427030 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.426943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfz6t\" (UniqueName: \"kubernetes.io/projected/f3dd827b-b347-46a4-b0b1-8c61ac63a401-kube-api-access-zfz6t\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.427030 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.426995 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3dd827b-b347-46a4-b0b1-8c61ac63a401-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.427381 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.427045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dd827b-b347-46a4-b0b1-8c61ac63a401-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.427381 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.427120 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dd827b-b347-46a4-b0b1-8c61ac63a401-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.527592 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.527555 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dd827b-b347-46a4-b0b1-8c61ac63a401-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.527762 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.527617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dd827b-b347-46a4-b0b1-8c61ac63a401-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.527762 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.527648 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfz6t\" (UniqueName: \"kubernetes.io/projected/f3dd827b-b347-46a4-b0b1-8c61ac63a401-kube-api-access-zfz6t\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.527762 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.527669 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3dd827b-b347-46a4-b0b1-8c61ac63a401-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.527762 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.527709 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dd827b-b347-46a4-b0b1-8c61ac63a401-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.527762 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.527758 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dd827b-b347-46a4-b0b1-8c61ac63a401-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.528174 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.528002 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dd827b-b347-46a4-b0b1-8c61ac63a401-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.528174 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.528021 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3dd827b-b347-46a4-b0b1-8c61ac63a401-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.528174 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.528170 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dd827b-b347-46a4-b0b1-8c61ac63a401-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.529929 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.529907 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dd827b-b347-46a4-b0b1-8c61ac63a401-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.530196 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.530178 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dd827b-b347-46a4-b0b1-8c61ac63a401-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.534546 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.534522 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfz6t\" (UniqueName: \"kubernetes.io/projected/f3dd827b-b347-46a4-b0b1-8c61ac63a401-kube-api-access-zfz6t\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk\" (UID: \"f3dd827b-b347-46a4-b0b1-8c61ac63a401\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.673790 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.673719 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:06.948223 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:06.948197 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk"] Sep 06 13:15:06.950679 ip-10-0-141-13 kubenswrapper[2580]: W0906 13:15:06.950641 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3dd827b_b347_46a4_b0b1_8c61ac63a401.slice/crio-552c877fd385d81cdc513cade4d3e8ec9f14b63c3bfab360fa528687bce8a97b WatchSource:0}: Error finding container 552c877fd385d81cdc513cade4d3e8ec9f14b63c3bfab360fa528687bce8a97b: Status 404 returned error can't find the container with id 552c877fd385d81cdc513cade4d3e8ec9f14b63c3bfab360fa528687bce8a97b Sep 06 13:15:07.599707 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:07.599661 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" event={"ID":"f3dd827b-b347-46a4-b0b1-8c61ac63a401","Type":"ContainerStarted","Data":"552c877fd385d81cdc513cade4d3e8ec9f14b63c3bfab360fa528687bce8a97b"} Sep 06 13:15:08.606325 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:08.606284 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" event={"ID":"f3dd827b-b347-46a4-b0b1-8c61ac63a401","Type":"ContainerStarted","Data":"0b3b4406b306eb236e957b36b3405743d739ba829c08554a8900a6ca63675c00"} Sep 06 13:15:08.621035 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:08.620992 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" podStartSLOduration=1.5926424909999999 podStartE2EDuration="2.620978628s" totalImagesPullingTime="1.028336137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 13:15:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 13:15:08.620576561 +0000 UTC m=+1718.362947716" watchObservedRunningTime="2026-09-06 13:15:08.620978628 +0000 UTC m=+1718.363349782" Sep 06 13:15:12.731042 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:12.731013 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:15:12.731513 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:15:12.731170 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:15:14.730731 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:14.730700 2580 scope.go:117] "RemoveContainer" containerID="ca943d738d732a1a45e9713ac7e0fe946920a651dc8a36d6a81f9a98cb4943ce" Sep 06 13:15:15.644068 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:15.643976 2580 generic.go:356] "Generic (PLEG): container finished" podID="57c0cce4-30b0-4904-af59-343ba6ea430a" containerID="78b0ad6c0e8557bfe0dd49f6cde7bdedc78ac1b4dfbd8db5ce71558b39353d55" exitCode=7 Sep 06 13:15:15.644068 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:15.644053 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4" event={"ID":"57c0cce4-30b0-4904-af59-343ba6ea430a","Type":"ContainerDied","Data":"78b0ad6c0e8557bfe0dd49f6cde7bdedc78ac1b4dfbd8db5ce71558b39353d55"} Sep 06 13:15:15.644327 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:15.644160 2580 scope.go:117] "RemoveContainer" containerID="ca943d738d732a1a45e9713ac7e0fe946920a651dc8a36d6a81f9a98cb4943ce" Sep 06 13:15:15.644401 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:15.644386 2580 scope.go:117] "RemoveContainer" containerID="78b0ad6c0e8557bfe0dd49f6cde7bdedc78ac1b4dfbd8db5ce71558b39353d55" Sep 06 13:15:15.644645 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:15:15.644625 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29811675-frnq4_odh-ai-gateway-infra(57c0cce4-30b0-4904-af59-343ba6ea430a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" Sep 06 13:15:16.666309 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:16.666258 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4"] Sep 06 13:15:16.674513 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:16.674469 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:16.674676 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:16.674538 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:16.687565 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:16.687536 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:16.794225 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:16.794203 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4" Sep 06 13:15:17.656915 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:17.656882 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4" Sep 06 13:15:17.657129 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:17.656921 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4" event={"ID":"57c0cce4-30b0-4904-af59-343ba6ea430a","Type":"ContainerDied","Data":"617e22b7132c6673286ce4e71d7ee6726aedc2e2250c2c70d60c8b25d65b3d37"} Sep 06 13:15:17.657129 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:17.656981 2580 scope.go:117] "RemoveContainer" containerID="78b0ad6c0e8557bfe0dd49f6cde7bdedc78ac1b4dfbd8db5ce71558b39353d55" Sep 06 13:15:17.668911 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:17.668886 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk" Sep 06 13:15:17.677895 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:17.677870 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4"] Sep 06 13:15:17.681493 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:17.681472 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811675-frnq4"] Sep 06 13:15:18.735087 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:18.735054 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" path="/var/lib/kubelet/pods/57c0cce4-30b0-4904-af59-343ba6ea430a/volumes" Sep 06 13:15:26.730770 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:26.730739 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:15:26.731380 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:15:26.730927 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:15:30.466039 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.466004 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f"] Sep 06 13:15:30.466539 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.466519 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" containerName="cleanup" Sep 06 13:15:30.466613 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.466543 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" containerName="cleanup" Sep 06 13:15:30.466613 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.466558 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" containerName="cleanup" Sep 06 13:15:30.466613 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.466566 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" containerName="cleanup" Sep 06 13:15:30.466613 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.466608 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" containerName="cleanup" Sep 06 13:15:30.466809 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.466617 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" containerName="cleanup" Sep 06 13:15:30.466809 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.466703 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" containerName="cleanup" Sep 06 13:15:30.466809 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.466715 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" containerName="cleanup" Sep 06 13:15:30.466809 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.466728 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57c0cce4-30b0-4904-af59-343ba6ea430a" containerName="cleanup" Sep 06 13:15:30.471698 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.471672 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.473882 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.473845 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 06 13:15:30.476702 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.476679 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f"] Sep 06 13:15:30.532949 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.532913 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3028b1f4-1793-4376-9988-d02551043d65-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.533135 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.532964 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gcqh\" (UniqueName: \"kubernetes.io/projected/3028b1f4-1793-4376-9988-d02551043d65-kube-api-access-6gcqh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.533135 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.533022 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3028b1f4-1793-4376-9988-d02551043d65-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.533135 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.533048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3028b1f4-1793-4376-9988-d02551043d65-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.533135 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.533123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3028b1f4-1793-4376-9988-d02551043d65-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.533362 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.533202 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3028b1f4-1793-4376-9988-d02551043d65-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.633770 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.633734 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3028b1f4-1793-4376-9988-d02551043d65-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.633953 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.633781 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3028b1f4-1793-4376-9988-d02551043d65-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.633953 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.633813 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3028b1f4-1793-4376-9988-d02551043d65-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.633953 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.633834 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gcqh\" (UniqueName: \"kubernetes.io/projected/3028b1f4-1793-4376-9988-d02551043d65-kube-api-access-6gcqh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.633953 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.633862 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3028b1f4-1793-4376-9988-d02551043d65-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.633953 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.633882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3028b1f4-1793-4376-9988-d02551043d65-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.634215 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.634150 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3028b1f4-1793-4376-9988-d02551043d65-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.634215 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.634191 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3028b1f4-1793-4376-9988-d02551043d65-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.634364 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.634343 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3028b1f4-1793-4376-9988-d02551043d65-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.636161 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.636132 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3028b1f4-1793-4376-9988-d02551043d65-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.636546 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.636529 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3028b1f4-1793-4376-9988-d02551043d65-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.640988 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.640959 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gcqh\" (UniqueName: \"kubernetes.io/projected/3028b1f4-1793-4376-9988-d02551043d65-kube-api-access-6gcqh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wdp6f\" (UID: \"3028b1f4-1793-4376-9988-d02551043d65\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:30.786307 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:30.786193 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:31.057050 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:31.057005 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f"] Sep 06 13:15:31.060376 ip-10-0-141-13 kubenswrapper[2580]: W0906 13:15:31.060334 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3028b1f4_1793_4376_9988_d02551043d65.slice/crio-bc84e1dc9aa5682402cfdb6fc072b097aafe64d04e3219e6103688d9f8ce9545 WatchSource:0}: Error finding container bc84e1dc9aa5682402cfdb6fc072b097aafe64d04e3219e6103688d9f8ce9545: Status 404 returned error can't find the container with id bc84e1dc9aa5682402cfdb6fc072b097aafe64d04e3219e6103688d9f8ce9545 Sep 06 13:15:31.722395 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:31.722357 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" event={"ID":"3028b1f4-1793-4376-9988-d02551043d65","Type":"ContainerStarted","Data":"68ac7844e159d1fe957f800fa47a1886f6f879926a9763933869b04dd91f23e5"} Sep 06 13:15:31.722774 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:31.722439 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" event={"ID":"3028b1f4-1793-4376-9988-d02551043d65","Type":"ContainerStarted","Data":"bc84e1dc9aa5682402cfdb6fc072b097aafe64d04e3219e6103688d9f8ce9545"} Sep 06 13:15:38.730800 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:38.730768 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:15:38.731361 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:15:38.730997 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:15:40.787353 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:40.787309 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:40.787798 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:40.787371 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:40.800034 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:40.800006 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:40.814631 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:40.814579 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" podStartSLOduration=10.643018812 podStartE2EDuration="10.814562719s" totalImagesPullingTime="171.543907ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 13:15:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 13:15:31.738248222 +0000 UTC m=+1741.480619377" watchObservedRunningTime="2026-09-06 13:15:40.814562719 +0000 UTC m=+1750.556933875" Sep 06 13:15:41.785072 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:41.785036 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wdp6f" Sep 06 13:15:52.163242 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.163204 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl"] Sep 06 13:15:52.166687 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.166664 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.168965 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.168944 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 06 13:15:52.177373 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.177341 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl"] Sep 06 13:15:52.217077 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.217037 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.217242 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.217111 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.217242 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.217132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwfhr\" (UniqueName: \"kubernetes.io/projected/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-kube-api-access-rwfhr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.217242 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.217158 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.217242 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.217238 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.217415 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.217282 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.317979 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.317935 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.317979 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.317981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwfhr\" (UniqueName: \"kubernetes.io/projected/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-kube-api-access-rwfhr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.318223 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.318003 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.318223 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.318046 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.318223 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.318066 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.318223 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.318085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.318527 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.318498 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.318591 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.318506 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.318591 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.318560 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.320561 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.320537 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.320731 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.320709 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.325770 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.325743 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwfhr\" (UniqueName: \"kubernetes.io/projected/9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5-kube-api-access-rwfhr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hq8nl\" (UID: \"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.478458 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.478362 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:15:52.730342 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.730229 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:15:52.730552 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:15:52.730519 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:15:52.748176 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.748149 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl"] Sep 06 13:15:52.750647 ip-10-0-141-13 kubenswrapper[2580]: W0906 13:15:52.750612 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a82279a_f82f_4a1e_aec5_cbf5cfb29dc5.slice/crio-0fc02f9cf0e42f4d1f7cd255bb74ed99ec4c2e1c944d18757913123d58d8979c WatchSource:0}: Error finding container 0fc02f9cf0e42f4d1f7cd255bb74ed99ec4c2e1c944d18757913123d58d8979c: Status 404 returned error can't find the container with id 0fc02f9cf0e42f4d1f7cd255bb74ed99ec4c2e1c944d18757913123d58d8979c Sep 06 13:15:52.821879 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:52.821843 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" event={"ID":"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5","Type":"ContainerStarted","Data":"0fc02f9cf0e42f4d1f7cd255bb74ed99ec4c2e1c944d18757913123d58d8979c"} Sep 06 13:15:53.828353 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:53.828320 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" event={"ID":"9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5","Type":"ContainerStarted","Data":"fcd83a5f74b27860f077ee1a65888dd272a121e0cadaf5874c56e26d865298c5"} Sep 06 13:15:53.844861 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:15:53.844809 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" podStartSLOduration=1.674455376 podStartE2EDuration="1.844792179s" totalImagesPullingTime="170.336803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 13:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 13:15:53.843875719 +0000 UTC m=+1763.586246897" watchObservedRunningTime="2026-09-06 13:15:53.844792179 +0000 UTC m=+1763.587163334" Sep 06 13:16:02.069851 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.069815 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4"] Sep 06 13:16:02.073444 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.073422 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.075588 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.075557 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 06 13:16:02.082512 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.082486 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4"] Sep 06 13:16:02.209749 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.209706 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/185a98ee-283d-4508-b701-347112847ac8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.209749 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.209750 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5xhc\" (UniqueName: \"kubernetes.io/projected/185a98ee-283d-4508-b701-347112847ac8-kube-api-access-x5xhc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.209975 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.209836 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/185a98ee-283d-4508-b701-347112847ac8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.209975 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.209869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/185a98ee-283d-4508-b701-347112847ac8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.209975 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.209958 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/185a98ee-283d-4508-b701-347112847ac8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.210074 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.210007 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/185a98ee-283d-4508-b701-347112847ac8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.311291 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.311222 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/185a98ee-283d-4508-b701-347112847ac8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.311465 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.311306 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5xhc\" (UniqueName: \"kubernetes.io/projected/185a98ee-283d-4508-b701-347112847ac8-kube-api-access-x5xhc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.311465 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.311352 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/185a98ee-283d-4508-b701-347112847ac8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.311465 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.311386 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/185a98ee-283d-4508-b701-347112847ac8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.311465 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.311426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/185a98ee-283d-4508-b701-347112847ac8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.311465 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.311457 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/185a98ee-283d-4508-b701-347112847ac8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.311869 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.311833 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/185a98ee-283d-4508-b701-347112847ac8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.311869 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.311867 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/185a98ee-283d-4508-b701-347112847ac8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.312072 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.311900 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/185a98ee-283d-4508-b701-347112847ac8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.313586 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.313566 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/185a98ee-283d-4508-b701-347112847ac8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.313922 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.313904 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/185a98ee-283d-4508-b701-347112847ac8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.318840 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.318809 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5xhc\" (UniqueName: \"kubernetes.io/projected/185a98ee-283d-4508-b701-347112847ac8-kube-api-access-x5xhc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4\" (UID: \"185a98ee-283d-4508-b701-347112847ac8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.386754 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.386662 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:02.478880 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.478846 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:16:02.479056 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.478938 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:16:02.491329 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.491298 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:16:02.659351 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.659327 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4"] Sep 06 13:16:02.661691 ip-10-0-141-13 kubenswrapper[2580]: W0906 13:16:02.661663 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod185a98ee_283d_4508_b701_347112847ac8.slice/crio-87b8d56e3daaf22f395f8b5e5dd1281e90ec21440eff0bd7684b5a792909c3ed WatchSource:0}: Error finding container 87b8d56e3daaf22f395f8b5e5dd1281e90ec21440eff0bd7684b5a792909c3ed: Status 404 returned error can't find the container with id 87b8d56e3daaf22f395f8b5e5dd1281e90ec21440eff0bd7684b5a792909c3ed Sep 06 13:16:02.869343 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.869225 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" event={"ID":"185a98ee-283d-4508-b701-347112847ac8","Type":"ContainerStarted","Data":"87b8d56e3daaf22f395f8b5e5dd1281e90ec21440eff0bd7684b5a792909c3ed"} Sep 06 13:16:02.884578 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:02.884553 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hq8nl" Sep 06 13:16:03.875140 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:03.875101 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" event={"ID":"185a98ee-283d-4508-b701-347112847ac8","Type":"ContainerStarted","Data":"c4cc1976fbd43168ac7553368af33eff3822be38e5f62733a663ceadd9e28c22"} Sep 06 13:16:03.890459 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:03.890406 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" podStartSLOduration=1.728587643 podStartE2EDuration="1.890390128s" totalImagesPullingTime="161.802485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 13:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 13:16:03.889636701 +0000 UTC m=+1773.632007858" watchObservedRunningTime="2026-09-06 13:16:03.890390128 +0000 UTC m=+1773.632761283" Sep 06 13:16:04.731089 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:04.731054 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:16:04.731367 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:16:04.731296 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:16:12.387481 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:12.387438 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:12.387944 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:12.387520 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:12.400015 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:12.399987 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:12.932502 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:12.932467 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4" Sep 06 13:16:15.730920 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:15.730882 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:16:15.731405 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:16:15.731133 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:16:29.730800 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:29.730762 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:16:29.731231 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:29.731155 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 13:16:30.823656 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:30.823560 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 13:16:30.828180 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:30.828158 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 13:16:31.007081 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:31.007050 2580 generic.go:356] "Generic (PLEG): container finished" podID="b18ffbb6-5f3f-4715-be72-4195c934563d" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" exitCode=1 Sep 06 13:16:31.007245 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:31.007099 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" event={"ID":"b18ffbb6-5f3f-4715-be72-4195c934563d","Type":"ContainerDied","Data":"63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97"} Sep 06 13:16:31.007245 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:31.007172 2580 scope.go:117] "RemoveContainer" containerID="89e581f9cac41cca9bb59d364de6fa427ef8d761432e999c704bece4949c00c9" Sep 06 13:16:31.007511 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:31.007493 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:16:31.007741 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:16:31.007716 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:16:32.739688 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:32.739662 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 13:16:32.740068 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:32.739924 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:16:32.740068 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:16:32.740060 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:16:37.739441 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:37.739404 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" Sep 06 13:16:37.739896 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:37.739776 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:16:37.739998 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:16:37.739975 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:16:48.730691 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:16:48.730652 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:16:48.731159 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:16:48.730918 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:17:01.730126 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:17:01.730095 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:17:01.730540 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:17:01.730314 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:17:12.731128 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:17:12.731093 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:17:12.731622 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:17:12.731293 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:17:24.730556 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:17:24.730520 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:17:24.730961 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:17:24.730745 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:17:37.730157 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:17:37.730126 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:17:37.730681 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:17:37.730408 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:17:49.729959 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:17:49.729931 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:17:49.730470 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:17:49.730109 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:18:00.733099 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:18:00.733014 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:18:00.733498 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:18:00.733235 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:18:13.730718 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:18:13.730685 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:18:13.731228 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:18:13.730928 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:18:26.730161 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:18:26.730136 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:18:26.730612 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:18:26.730343 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:18:37.729948 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:18:37.729918 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:18:37.730450 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:18:37.730158 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:18:49.730886 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:18:49.730855 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:18:49.731325 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:18:49.731042 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:19:03.730905 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:03.730872 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:19:03.731348 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:19:03.731066 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:19:17.730629 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:17.730599 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:19:17.731039 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:19:17.730788 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:19:28.730836 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:28.730741 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:19:28.731300 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:19:28.730936 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:19:31.565949 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:31.565916 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-777d65df8-f4lkl_a128b654-e51a-42bf-94e9-6e9bb97ed624/manager/0.log" Sep 06 13:19:31.904594 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:31.904508 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b4f78958f-lqfgj_5db20faf-04c4-4a70-b5a3-228673e18e11/copy-manifests/0.log" Sep 06 13:19:31.915605 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:31.915577 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b4f78958f-lqfgj_5db20faf-04c4-4a70-b5a3-228673e18e11/manager/0.log" Sep 06 13:19:32.471687 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:32.471659 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-f57cbc8cb-p87c9_b18ffbb6-5f3f-4715-be72-4195c934563d/maas-api/9.log" Sep 06 13:19:32.585836 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:32.585809 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-j4862_f43fa617-d08a-437f-b0b5-833bcbcda082/postgres/0.log" Sep 06 13:19:34.379017 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:34.378982 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r_930468da-ae0a-4fad-b360-6c37df34fe41/istio-proxy/0.log" Sep 06 13:19:34.845143 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:34.845118 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6gs8m_5178826f-8d7f-4be4-871b-528202e9198d/istio-proxy/1.log" Sep 06 13:19:34.957455 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:34.957426 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-zhskz_8d6d8874-fa5c-433e-8720-4682f258089c/payload-pre-processing/0.log" Sep 06 13:19:35.188126 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:35.188039 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-659dc76f98-xl7g6_75be41fd-be89-4ca5-9bab-ac64e74295ba/router/0.log" Sep 06 13:19:35.535894 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:35.535818 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmxk_f3dd827b-b347-46a4-b0b1-8c61ac63a401/main/0.log" Sep 06 13:19:35.646408 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:35.646380 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-wdp6f_3028b1f4-1793-4376-9988-d02551043d65/main/0.log" Sep 06 13:19:35.864799 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:35.864768 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hq8nl_9a82279a-f82f-4a1e-aec5-cbf5cfb29dc5/main/0.log" Sep 06 13:19:35.973123 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:35.973094 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fr59j4_185a98ee-283d-4508-b701-347112847ac8/main/0.log" Sep 06 13:19:43.729961 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:43.729929 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:19:43.730385 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:19:43.730108 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:19:44.898827 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:44.898799 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-d9f5r_5823a60b-e18f-4f3c-b8f5-07cf5ce58135/global-pull-secret-syncer/0.log" Sep 06 13:19:45.015768 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:45.015738 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dhmsm_ff818479-7d03-4a64-8e09-2b094c4011a1/konnectivity-agent/0.log" Sep 06 13:19:45.055557 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:45.055529 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-13.ec2.internal_656e14dae41de34e23352cc32ee02da6/haproxy/0.log" Sep 06 13:19:51.914217 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:51.914183 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-9gph2_b8dd16fa-c99c-4c68-b472-4927f82d5e5a/cluster-monitoring-operator/0.log" Sep 06 13:19:52.180588 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:52.180512 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwz9w_d2b2e3af-d7c9-49f0-a5d0-bfc985417b52/node-exporter/0.log" Sep 06 13:19:52.200946 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:52.200919 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwz9w_d2b2e3af-d7c9-49f0-a5d0-bfc985417b52/kube-rbac-proxy/0.log" Sep 06 13:19:52.229750 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:52.229726 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwz9w_d2b2e3af-d7c9-49f0-a5d0-bfc985417b52/init-textfile/0.log" Sep 06 13:19:54.115827 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.115798 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-d2c2j_c28de569-a5a3-46dc-b764-7244ecb5b380/networking-console-plugin/0.log" Sep 06 13:19:54.486113 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.486033 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc"] Sep 06 13:19:54.489925 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.489904 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.492134 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.492110 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8csgg\"/\"openshift-service-ca.crt\"" Sep 06 13:19:54.492281 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.492158 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8csgg\"/\"kube-root-ca.crt\"" Sep 06 13:19:54.492281 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.492213 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8csgg\"/\"default-dockercfg-4482s\"" Sep 06 13:19:54.499254 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.499229 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc"] Sep 06 13:19:54.593006 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.592967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/26d99dc1-ed17-4521-ae16-7ae9c05cd758-podres\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.593006 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.593007 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26d99dc1-ed17-4521-ae16-7ae9c05cd758-sys\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.593294 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.593032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5qlk\" (UniqueName: \"kubernetes.io/projected/26d99dc1-ed17-4521-ae16-7ae9c05cd758-kube-api-access-d5qlk\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.593294 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.593157 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/26d99dc1-ed17-4521-ae16-7ae9c05cd758-proc\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.593294 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.593256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26d99dc1-ed17-4521-ae16-7ae9c05cd758-lib-modules\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.694123 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.694086 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/26d99dc1-ed17-4521-ae16-7ae9c05cd758-proc\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.694335 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.694148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26d99dc1-ed17-4521-ae16-7ae9c05cd758-lib-modules\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.694335 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.694171 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/26d99dc1-ed17-4521-ae16-7ae9c05cd758-podres\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.694335 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.694185 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26d99dc1-ed17-4521-ae16-7ae9c05cd758-sys\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.694335 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.694202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5qlk\" (UniqueName: \"kubernetes.io/projected/26d99dc1-ed17-4521-ae16-7ae9c05cd758-kube-api-access-d5qlk\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.694335 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.694232 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/26d99dc1-ed17-4521-ae16-7ae9c05cd758-proc\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.694335 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.694313 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26d99dc1-ed17-4521-ae16-7ae9c05cd758-sys\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.694564 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.694358 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26d99dc1-ed17-4521-ae16-7ae9c05cd758-lib-modules\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.694564 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.694374 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/26d99dc1-ed17-4521-ae16-7ae9c05cd758-podres\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.700920 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.700897 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5qlk\" (UniqueName: \"kubernetes.io/projected/26d99dc1-ed17-4521-ae16-7ae9c05cd758-kube-api-access-d5qlk\") pod \"perf-node-gather-daemonset-pcnbc\" (UID: \"26d99dc1-ed17-4521-ae16-7ae9c05cd758\") " pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:54.737119 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.737026 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/1.log" Sep 06 13:19:54.741716 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.741693 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2vfhs_8750b827-635a-4805-817c-45a970d9fc81/console-operator/2.log" Sep 06 13:19:54.800650 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:54.800619 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:55.067982 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:55.067953 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc"] Sep 06 13:19:55.731063 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:55.731036 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:19:55.731493 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:19:55.731207 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:19:55.902510 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:55.902485 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-8zhfs_9c2987eb-10ff-4823-966e-7e65d410fc56/volume-data-source-validator/0.log" Sep 06 13:19:55.959222 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:55.959183 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" event={"ID":"26d99dc1-ed17-4521-ae16-7ae9c05cd758","Type":"ContainerStarted","Data":"74dbd300428b2c2466a3e514cd0a64efad3765134bad605eb3ed58176f60ecfd"} Sep 06 13:19:55.959403 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:55.959244 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:19:55.959403 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:55.959282 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" event={"ID":"26d99dc1-ed17-4521-ae16-7ae9c05cd758","Type":"ContainerStarted","Data":"1aad70d16cad672bc26e333c25755eba3743c8bf19bfdc1a951fd5acf7ec7ac5"} Sep 06 13:19:55.973819 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:55.973776 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" podStartSLOduration=1.973761288 podStartE2EDuration="1.973761288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 13:19:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 13:19:55.972422049 +0000 UTC m=+2005.714793204" watchObservedRunningTime="2026-09-06 13:19:55.973761288 +0000 UTC m=+2005.716132442" Sep 06 13:19:56.879048 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:56.879019 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fgcpf_c34f36a1-7b51-43f1-a481-e5331b2ee206/dns/0.log" Sep 06 13:19:56.897937 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:56.897900 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fgcpf_c34f36a1-7b51-43f1-a481-e5331b2ee206/kube-rbac-proxy/0.log" Sep 06 13:19:56.998583 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:56.998554 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-89gcv_3568c869-d959-469d-be7e-c73faa22d4d9/dns-node-resolver/0.log" Sep 06 13:19:57.577561 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:57.577529 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qrd8m_5c30ce1d-31d0-4c59-b917-59e79bc7093f/node-ca/0.log" Sep 06 13:19:58.478576 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:58.478521 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r_930468da-ae0a-4fad-b360-6c37df34fe41/istio-proxy/0.log" Sep 06 13:19:58.608071 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:58.608037 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6gs8m_5178826f-8d7f-4be4-871b-528202e9198d/istio-proxy/1.log" Sep 06 13:19:58.608511 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:58.608496 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6gs8m_5178826f-8d7f-4be4-871b-528202e9198d/istio-proxy/0.log" Sep 06 13:19:58.624801 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:58.624778 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-zhskz_8d6d8874-fa5c-433e-8720-4682f258089c/payload-pre-processing/0.log" Sep 06 13:19:58.668996 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:58.668960 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-659dc76f98-xl7g6_75be41fd-be89-4ca5-9bab-ac64e74295ba/router/0.log" Sep 06 13:19:59.294351 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:59.294321 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h8tgn_f09a5d91-5be0-4cff-8388-d3f33dfb0391/serve-healthcheck-canary/0.log" Sep 06 13:19:59.731183 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:59.731158 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-996bm_b93d75ad-6183-460c-9652-c502a31ba231/insights-operator/0.log" Sep 06 13:19:59.732710 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:59.732692 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-996bm_b93d75ad-6183-460c-9652-c502a31ba231/insights-operator/1.log" Sep 06 13:19:59.749955 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:59.749930 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-462kn_5d654b09-beec-407d-b4b7-cc802e221d43/kube-rbac-proxy/0.log" Sep 06 13:19:59.767414 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:59.767397 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-462kn_5d654b09-beec-407d-b4b7-cc802e221d43/exporter/0.log" Sep 06 13:19:59.784906 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:19:59.784890 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-462kn_5d654b09-beec-407d-b4b7-cc802e221d43/extractor/0.log" Sep 06 13:20:01.973507 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:01.973477 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8csgg/perf-node-gather-daemonset-pcnbc" Sep 06 13:20:01.997997 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:01.997967 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-777d65df8-f4lkl_a128b654-e51a-42bf-94e9-6e9bb97ed624/manager/0.log" Sep 06 13:20:02.124205 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:02.124173 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b4f78958f-lqfgj_5db20faf-04c4-4a70-b5a3-228673e18e11/manager/0.log" Sep 06 13:20:02.138831 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:02.138806 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b4f78958f-lqfgj_5db20faf-04c4-4a70-b5a3-228673e18e11/copy-manifests/0.log" Sep 06 13:20:03.475727 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:03.475679 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8566865f66-tkcdt_78979b32-c785-4a5e-a1ba-0a0e66320476/manager/0.log" Sep 06 13:20:05.151200 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:05.151170 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r_930468da-ae0a-4fad-b360-6c37df34fe41/istio-proxy/0.log" Sep 06 13:20:05.253327 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:05.253296 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6gs8m_5178826f-8d7f-4be4-871b-528202e9198d/istio-proxy/1.log" Sep 06 13:20:05.253682 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:05.253664 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6gs8m_5178826f-8d7f-4be4-871b-528202e9198d/istio-proxy/0.log" Sep 06 13:20:05.262440 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:05.262416 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-zhskz_8d6d8874-fa5c-433e-8720-4682f258089c/payload-pre-processing/0.log" Sep 06 13:20:05.301835 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:05.301807 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-659dc76f98-xl7g6_75be41fd-be89-4ca5-9bab-ac64e74295ba/router/0.log" Sep 06 13:20:06.509943 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:06.509854 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-d226r_930468da-ae0a-4fad-b360-6c37df34fe41/istio-proxy/0.log" Sep 06 13:20:06.619975 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:06.619944 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6gs8m_5178826f-8d7f-4be4-871b-528202e9198d/istio-proxy/1.log" Sep 06 13:20:06.620362 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:06.620338 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6gs8m_5178826f-8d7f-4be4-871b-528202e9198d/istio-proxy/0.log" Sep 06 13:20:06.635395 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:06.635373 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-zhskz_8d6d8874-fa5c-433e-8720-4682f258089c/payload-pre-processing/0.log" Sep 06 13:20:06.671594 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:06.671568 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-659dc76f98-xl7g6_75be41fd-be89-4ca5-9bab-ac64e74295ba/router/0.log" Sep 06 13:20:08.777958 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:08.777924 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-799wl_0e03a1d0-49d0-4899-86ce-779bf37dd7a6/migrator/0.log" Sep 06 13:20:08.794946 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:08.794921 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-799wl_0e03a1d0-49d0-4899-86ce-779bf37dd7a6/graceful-termination/0.log" Sep 06 13:20:09.136990 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:09.136949 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-s6ght_ab6850db-dfc3-47f9-b91b-54757857d601/kube-storage-version-migrator-operator/1.log" Sep 06 13:20:09.138000 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:09.137975 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-s6ght_ab6850db-dfc3-47f9-b91b-54757857d601/kube-storage-version-migrator-operator/0.log" Sep 06 13:20:09.730830 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:09.730788 2580 scope.go:117] "RemoveContainer" containerID="63c3f0d60c6959ef9a17bee2f2e60b2c49aca3ae8a236a2045ccfaaa0b879c97" Sep 06 13:20:09.731019 ip-10-0-141-13 kubenswrapper[2580]: E0906 13:20:09.730993 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-f57cbc8cb-p87c9_odh-ai-gateway-infra(b18ffbb6-5f3f-4715-be72-4195c934563d)\"" pod="odh-ai-gateway-infra/maas-api-f57cbc8cb-p87c9" podUID="b18ffbb6-5f3f-4715-be72-4195c934563d" Sep 06 13:20:10.303587 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:10.303556 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmxfr_55fb0a7b-617b-477c-ae3a-2c6cf1b33480/kube-multus-additional-cni-plugins/0.log" Sep 06 13:20:10.322020 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:10.321975 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmxfr_55fb0a7b-617b-477c-ae3a-2c6cf1b33480/egress-router-binary-copy/0.log" Sep 06 13:20:10.339667 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:10.339638 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmxfr_55fb0a7b-617b-477c-ae3a-2c6cf1b33480/cni-plugins/0.log" Sep 06 13:20:10.357009 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:10.356983 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmxfr_55fb0a7b-617b-477c-ae3a-2c6cf1b33480/bond-cni-plugin/0.log" Sep 06 13:20:10.374966 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:10.374937 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmxfr_55fb0a7b-617b-477c-ae3a-2c6cf1b33480/routeoverride-cni/0.log" Sep 06 13:20:10.392701 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:10.392667 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmxfr_55fb0a7b-617b-477c-ae3a-2c6cf1b33480/whereabouts-cni-bincopy/0.log" Sep 06 13:20:10.412742 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:10.412712 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmxfr_55fb0a7b-617b-477c-ae3a-2c6cf1b33480/whereabouts-cni/0.log" Sep 06 13:20:10.583674 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:10.583592 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rprqz_987932ec-426e-496d-9da1-506be3f16b92/kube-multus/0.log" Sep 06 13:20:10.648150 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:10.648124 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5t886_42d02b82-c6e5-4400-b240-dd933ab712a1/network-metrics-daemon/0.log" Sep 06 13:20:10.665004 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:10.664978 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5t886_42d02b82-c6e5-4400-b240-dd933ab712a1/kube-rbac-proxy/0.log" Sep 06 13:20:11.548217 ip-10-0-141-13 kubenswrapper[2580]: I0906 13:20:11.548188 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4p8dj_a8776eb0-c74c-4194-82f9-23a98a1fe812/ovn-controller/0.log"