Sep 13 19:04:08.501588 ip-10-0-129-160 systemd[1]: Starting Kubernetes Kubelet... Sep 13 19:04:08.917228 ip-10-0-129-160 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 13 19:04:08.917228 ip-10-0-129-160 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 13 19:04:08.917228 ip-10-0-129-160 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 13 19:04:08.917228 ip-10-0-129-160 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 13 19:04:08.917228 ip-10-0-129-160 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 13 19:04:08.918553 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.918458 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 13 19:04:08.922565 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922539 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 13 19:04:08.922565 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922565 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 13 19:04:08.922565 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922569 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922572 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922576 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922578 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922581 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922584 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922586 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922589 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922592 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922595 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922597 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922600 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922603 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922605 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922608 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922611 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922613 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922616 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922619 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922622 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 13 19:04:08.922677 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922624 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922627 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922636 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922639 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922642 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922645 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922647 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922650 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922653 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922655 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922658 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922661 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922664 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922668 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922670 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922673 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922676 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922678 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922681 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 13 19:04:08.923158 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922684 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922687 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922690 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922693 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922695 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922698 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922702 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922704 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922707 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922710 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922712 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922716 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922718 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922721 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922724 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922727 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922729 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922731 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922734 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922737 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 13 19:04:08.923652 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922739 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922742 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922745 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922748 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922750 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922754 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922757 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922760 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922762 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922766 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922769 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922772 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922774 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922777 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922781 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922784 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922787 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922791 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922795 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 13 19:04:08.924155 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922797 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922801 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922804 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922808 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922811 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922814 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922817 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922819 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922822 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922825 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922827 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922830 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922833 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922836 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922838 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922841 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922844 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922847 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.922850 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923608 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 13 19:04:08.924639 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923616 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923619 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923622 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923625 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923628 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923630 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923633 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923636 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923640 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923643 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923645 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923648 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923651 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923654 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923657 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923659 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923662 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923665 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923668 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923670 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 13 19:04:08.925250 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923673 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923676 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923678 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923681 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923683 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923686 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923689 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923691 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923695 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923698 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923701 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923705 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923711 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923715 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923719 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923723 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923727 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923731 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923735 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 13 19:04:08.925763 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923739 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923743 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923747 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923752 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923756 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923760 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923764 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923769 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923772 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923779 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923784 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923788 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923792 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923797 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923801 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923806 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923811 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923815 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923822 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923828 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 13 19:04:08.926230 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923834 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923839 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923844 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923849 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923854 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923860 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923864 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923869 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923873 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923878 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923883 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923888 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923892 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923896 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923900 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923905 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923910 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923914 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923918 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 13 19:04:08.926731 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923922 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923924 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923927 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923930 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923933 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923936 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923938 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923941 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923944 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923947 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923949 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923952 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923954 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923957 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923960 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923963 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923966 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923968 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923971 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.923975 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 13 19:04:08.927212 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924415 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924422 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924425 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924428 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924431 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924434 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924437 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924439 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924443 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924445 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924448 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924451 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924453 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924456 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924458 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924461 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924464 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924467 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924470 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924473 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 13 19:04:08.927728 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924475 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924478 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924481 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924483 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924486 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924489 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924492 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924494 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924497 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924500 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924502 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924505 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924507 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924510 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924513 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924516 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924518 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924521 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924523 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924526 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 13 19:04:08.928228 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924528 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924532 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924534 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924537 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924540 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924542 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924545 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924547 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924550 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924552 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924555 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924557 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924560 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924562 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924565 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924567 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924570 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924573 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924576 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924578 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 13 19:04:08.928741 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924581 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924584 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924586 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924588 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924591 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924593 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924596 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924598 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924601 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924605 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924609 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924613 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924616 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924619 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924627 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924630 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924633 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924636 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924639 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 13 19:04:08.929265 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924643 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924645 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924649 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924652 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924655 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924657 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924660 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924663 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924665 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924668 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924671 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924674 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924676 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924679 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924683 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924687 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924689 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924695 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924698 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.924701 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 13 19:04:08.929734 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924792 2563 flags.go:64] FLAG: --address="0.0.0.0" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924839 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924847 2563 flags.go:64] FLAG: --anonymous-auth="true" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924856 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924871 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924875 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924879 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924884 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924907 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924913 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924918 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924922 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924925 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924928 2563 flags.go:64] FLAG: --cgroup-root="" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924932 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924935 2563 flags.go:64] FLAG: --client-ca-file="" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924938 2563 flags.go:64] FLAG: --cloud-provider="external" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924941 2563 flags.go:64] FLAG: --cluster-dns="[]" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924949 2563 flags.go:64] FLAG: --cluster-domain="" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924952 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924955 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924959 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924962 2563 flags.go:64] FLAG: --container-log-max-files="5" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924967 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 13 19:04:08.930280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924970 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924973 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924977 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924980 2563 flags.go:64] FLAG: --contention-profiling="false" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924983 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924989 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924992 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.924995 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925000 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925003 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925006 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925015 2563 flags.go:64] FLAG: --enable-load-reader="false" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925018 2563 flags.go:64] FLAG: --enable-server="true" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925021 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925028 2563 flags.go:64] FLAG: --event-burst="100" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925031 2563 flags.go:64] FLAG: --event-qps="50" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925034 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925038 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925040 2563 flags.go:64] FLAG: --eviction-hard="" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925044 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925047 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925050 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925053 2563 flags.go:64] FLAG: --eviction-soft="" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925056 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925059 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 13 19:04:08.930893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925063 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925066 2563 flags.go:64] FLAG: --experimental-mounter-path="" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925069 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925072 2563 flags.go:64] FLAG: --fail-swap-on="true" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925074 2563 flags.go:64] FLAG: --feature-gates="" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925078 2563 flags.go:64] FLAG: --file-check-frequency="20s" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925081 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925084 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925088 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925091 2563 flags.go:64] FLAG: --healthz-port="10248" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925094 2563 flags.go:64] FLAG: --help="false" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925097 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-129-160.ec2.internal" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925101 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925104 2563 flags.go:64] FLAG: --http-check-frequency="20s" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925107 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925110 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925114 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925117 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925120 2563 flags.go:64] FLAG: --image-service-endpoint="" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925123 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925125 2563 flags.go:64] FLAG: --kube-api-burst="100" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925128 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925131 2563 flags.go:64] FLAG: --kube-api-qps="50" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925134 2563 flags.go:64] FLAG: --kube-reserved="" Sep 13 19:04:08.931530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925137 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925140 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925143 2563 flags.go:64] FLAG: --kubelet-cgroups="" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925146 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925149 2563 flags.go:64] FLAG: --lock-file="" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925152 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925155 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925158 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925164 2563 flags.go:64] FLAG: --log-json-split-stream="false" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925167 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925170 2563 flags.go:64] FLAG: --log-text-split-stream="false" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925173 2563 flags.go:64] FLAG: --logging-format="text" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925176 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925180 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925183 2563 flags.go:64] FLAG: --manifest-url="" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925185 2563 flags.go:64] FLAG: --manifest-url-header="" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925191 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925194 2563 flags.go:64] FLAG: --max-open-files="1000000" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925198 2563 flags.go:64] FLAG: --max-pods="110" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925201 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925206 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925209 2563 flags.go:64] FLAG: --memory-manager-policy="None" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925212 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925215 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925218 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 13 19:04:08.932118 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925221 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925246 2563 flags.go:64] FLAG: --node-status-max-images="50" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925251 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925256 2563 flags.go:64] FLAG: --oom-score-adj="-999" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925261 2563 flags.go:64] FLAG: --pod-cidr="" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925265 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925273 2563 flags.go:64] FLAG: --pod-manifest-path="" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925276 2563 flags.go:64] FLAG: --pod-max-pids="-1" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925279 2563 flags.go:64] FLAG: --pods-per-core="0" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925286 2563 flags.go:64] FLAG: --port="10250" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925289 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925292 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08138a7547ae99471" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925295 2563 flags.go:64] FLAG: --qos-reserved="" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925298 2563 flags.go:64] FLAG: --read-only-port="10255" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925301 2563 flags.go:64] FLAG: --register-node="true" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925304 2563 flags.go:64] FLAG: --register-with-taints="" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925308 2563 flags.go:64] FLAG: --registry-burst="10" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925318 2563 flags.go:64] FLAG: --registry-qps="5" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925321 2563 flags.go:64] FLAG: --reserved-cpus="" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925324 2563 flags.go:64] FLAG: --reserved-memory="" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925328 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925331 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925334 2563 flags.go:64] FLAG: --rotate-certificates="false" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925337 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925340 2563 flags.go:64] FLAG: --runonce="false" Sep 13 19:04:08.932917 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925343 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925347 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925351 2563 flags.go:64] FLAG: --seccomp-default="false" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925354 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925357 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925361 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925364 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925368 2563 flags.go:64] FLAG: --storage-driver-password="root" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925371 2563 flags.go:64] FLAG: --storage-driver-secure="false" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925374 2563 flags.go:64] FLAG: --storage-driver-table="stats" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925377 2563 flags.go:64] FLAG: --storage-driver-user="root" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925380 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925384 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925387 2563 flags.go:64] FLAG: --system-cgroups="" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925390 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925396 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925399 2563 flags.go:64] FLAG: --tls-cert-file="" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925409 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925414 2563 flags.go:64] FLAG: --tls-min-version="" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925417 2563 flags.go:64] FLAG: --tls-private-key-file="" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925420 2563 flags.go:64] FLAG: --topology-manager-policy="none" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925423 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925426 2563 flags.go:64] FLAG: --topology-manager-scope="container" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925429 2563 flags.go:64] FLAG: --v="2" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925435 2563 flags.go:64] FLAG: --version="false" Sep 13 19:04:08.933582 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925439 2563 flags.go:64] FLAG: --vmodule="" Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925444 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.925447 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925613 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925617 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925620 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925625 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925629 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925631 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925636 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925639 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925642 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925644 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925647 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925650 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925653 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925655 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925658 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925660 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 13 19:04:08.934220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925664 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925668 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925671 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925674 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925676 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925679 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925688 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925690 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925693 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925696 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925698 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925701 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925705 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925708 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925710 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925713 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925716 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925719 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925722 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925724 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 13 19:04:08.934712 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925727 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925730 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925734 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925737 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925739 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925742 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925744 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925747 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925750 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925752 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925755 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925757 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925760 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925762 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925765 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925768 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925770 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925773 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925776 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925784 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 13 19:04:08.935220 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925787 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925790 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925792 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925795 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925799 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925802 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925804 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925807 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925809 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925812 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925815 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925817 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925820 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925822 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925826 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925829 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925832 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925834 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925837 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925839 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 13 19:04:08.935733 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925842 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925845 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925847 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925850 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925852 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925855 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925857 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925860 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925862 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925865 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925867 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925870 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925873 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925876 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925878 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925881 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925884 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925887 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925889 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925892 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 13 19:04:08.936233 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925896 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925899 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.925903 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.926333 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.934369 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.934391 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.934415 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.934426 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934498 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934504 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934507 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934511 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934514 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934517 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934521 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 13 19:04:08.936869 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934524 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934527 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934530 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934533 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934536 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934539 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934542 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934545 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934548 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934551 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934556 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934560 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934564 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934568 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934572 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934577 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934581 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934584 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934587 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 13 19:04:08.937291 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934590 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934592 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934595 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934599 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934602 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934605 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934607 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934610 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934612 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934615 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934618 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934620 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934624 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934628 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934632 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934637 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934642 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934646 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934650 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 13 19:04:08.937811 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934652 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934656 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934659 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934662 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934664 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934667 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934670 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934672 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934675 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934678 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934681 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934684 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934686 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934689 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934691 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934694 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934697 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934700 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934703 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934706 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 13 19:04:08.938305 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934711 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934715 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934719 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934723 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934726 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934729 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934732 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934734 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934737 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934739 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934742 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934745 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934748 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934750 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934753 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934755 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934758 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934761 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934763 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934766 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 13 19:04:08.938937 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934769 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934772 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934775 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934778 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934780 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934783 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934786 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934790 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934795 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934799 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934803 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934805 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934808 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934812 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.934818 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 13 19:04:08.939479 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934958 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934965 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934969 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934972 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934976 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934979 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934983 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934987 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934990 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934993 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934995 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.934999 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935002 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935005 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935008 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935011 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935013 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935016 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935021 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 13 19:04:08.939876 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935026 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935030 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935034 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935037 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935039 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935042 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935045 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935047 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935050 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935053 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935056 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935059 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935061 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935064 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935067 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935069 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935072 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935074 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935077 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935080 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 13 19:04:08.940517 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935082 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935085 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935087 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935090 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935093 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935097 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935101 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935105 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935109 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935113 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935116 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935119 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935122 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935125 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935128 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935130 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935133 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935135 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935138 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935140 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 13 19:04:08.941019 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935143 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935146 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935150 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935152 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935155 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935157 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935161 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935164 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935166 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935169 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935171 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935177 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935182 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935187 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935190 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935193 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935195 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935198 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935200 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 13 19:04:08.941844 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935203 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935205 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935208 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935210 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935214 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935216 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935219 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935221 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935224 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935227 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935230 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935233 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935251 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935256 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935260 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935263 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935266 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935269 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935272 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 13 19:04:08.942515 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935275 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 13 19:04:08.943054 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:08.935277 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 13 19:04:08.943054 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.935283 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 13 19:04:08.943054 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.935437 2563 server.go:962] "Client rotation is on, will bootstrap in background" Sep 13 19:04:08.943054 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.937567 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 13 19:04:08.943054 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.938328 2563 server.go:1019] "Starting client certificate rotation" Sep 13 19:04:08.943054 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.938458 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 13 19:04:08.943054 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.938518 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 13 19:04:08.943054 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.940704 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 13 19:04:08.946730 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.946705 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 13 19:04:08.955058 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.955031 2563 log.go:25] "Validated CRI v1 runtime API" Sep 13 19:04:08.959987 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.959965 2563 log.go:25] "Validated CRI v1 image API" Sep 13 19:04:08.961788 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.961770 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 13 19:04:08.964021 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.963988 2563 fs.go:135] Filesystem UUIDs: map[578ca8dd-5a2c-40f7-8d44-0de99154f642:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e0518f77-0f85-4476-a690-ba7d52843e33:/dev/nvme0n1p4] Sep 13 19:04:08.964119 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.964020 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 13 19:04:08.968206 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.968181 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 13 19:04:08.970847 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.970684 2563 manager.go:216] Machine: {Timestamp:2026-09-13 19:04:08.969077583 +0000 UTC m=+0.350606606 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099994 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27198e5ef6a5fd1fbd18f785c8ea36 SystemUUID:ec27198e-5ef6-a5fd-1fbd-18f785c8ea36 BootID:6aa60fd5-8610-4d4d-88dc-fff76d0e7e8e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c3:57:51:1b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c3:57:51:1b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:89:c9:6d:44:e4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 13 19:04:08.970847 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.970839 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 13 19:04:08.971037 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.971022 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 13 19:04:08.974088 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.974045 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 13 19:04:08.974264 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.974093 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-160.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 13 19:04:08.974974 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.974959 2563 topology_manager.go:138] "Creating topology manager with none policy" Sep 13 19:04:08.975051 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.974975 2563 container_manager_linux.go:309] "Creating device plugin manager" Sep 13 19:04:08.975051 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.974984 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 13 19:04:08.976307 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.976290 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 13 19:04:08.977114 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.977099 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 13 19:04:08.979707 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.979691 2563 state_mem.go:36] "Initialized new in-memory state store" Sep 13 19:04:08.979840 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.979830 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 13 19:04:08.979873 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.979863 2563 kubelet.go:486] "Attempting to sync node with API server" Sep 13 19:04:08.979911 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.979874 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 13 19:04:08.979911 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.979889 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 13 19:04:08.979911 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.979904 2563 kubelet.go:398] "Adding apiserver pod source" Sep 13 19:04:08.980001 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.979913 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 13 19:04:08.980410 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.980387 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 13 19:04:08.980523 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.980431 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 13 19:04:08.982507 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.982475 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 13 19:04:08.984342 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.984327 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 13 19:04:08.984394 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.984358 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 13 19:04:08.985883 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.985864 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 13 19:04:08.985883 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.985885 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 13 19:04:08.986016 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.985891 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 13 19:04:08.986016 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.985898 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 13 19:04:08.986016 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.985906 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 13 19:04:08.986016 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.985912 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 13 19:04:08.986016 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.985931 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 13 19:04:08.986016 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.985937 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 13 19:04:08.986016 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.985944 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 13 19:04:08.986016 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.985950 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 13 19:04:08.986016 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.985959 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 13 19:04:08.986016 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.985964 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 13 19:04:08.986599 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.986585 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 13 19:04:08.986649 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.986601 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 13 19:04:08.988899 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:08.988875 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 13 19:04:08.989046 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.988926 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-160.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 13 19:04:08.989326 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:08.989255 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 13 19:04:08.991256 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.991225 2563 server.go:1268] "Started kubelet" Sep 13 19:04:08.991370 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.991282 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 13 19:04:08.991458 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.991405 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 13 19:04:08.991574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.991477 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 13 19:04:08.992281 ip-10-0-129-160 systemd[1]: Started Kubernetes Kubelet. Sep 13 19:04:08.993383 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:08.993343 2563 server.go:310] "Adding debug handlers to kubelet server" Sep 13 19:04:08.995923 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:08.995894 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 13 19:04:08.997612 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:08.996516 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-160.ec2.internal.18d4f6be4266d915 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-160.ec2.internal,UID:ip-10-0-129-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-160.ec2.internal,},FirstTimestamp:2026-09-13 19:04:08.991185173 +0000 UTC m=+0.372714199,LastTimestamp:2026-09-13 19:04:08.991185173 +0000 UTC m=+0.372714199,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-160.ec2.internal,}" Sep 13 19:04:09.017726 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.017698 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 13 19:04:09.021721 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.021698 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 13 19:04:09.021827 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.021733 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 13 19:04:09.021881 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.021858 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:09.022538 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.022449 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 13 19:04:09.022538 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.022466 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 13 19:04:09.022538 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.022449 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 13 19:04:09.022749 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.022697 2563 reconstruct.go:97] "Volume reconstruction finished" Sep 13 19:04:09.022749 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.022710 2563 reconciler.go:29] "Reconciler: start to sync state" Sep 13 19:04:09.022850 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.022751 2563 factory.go:55] Registering systemd factory Sep 13 19:04:09.022850 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.022766 2563 factory.go:223] Registration of the systemd container factory successfully Sep 13 19:04:09.023134 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.023115 2563 factory.go:153] Registering CRI-O factory Sep 13 19:04:09.023233 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.023138 2563 factory.go:223] Registration of the crio container factory successfully Sep 13 19:04:09.023233 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.023192 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 13 19:04:09.023233 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.023209 2563 factory.go:103] Registering Raw factory Sep 13 19:04:09.023233 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.023224 2563 manager.go:1196] Started watching for new ooms in manager Sep 13 19:04:09.023593 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.023580 2563 manager.go:349] Starting recovery of all containers Sep 13 19:04:09.023736 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.023711 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 13 19:04:09.023817 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.023733 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 13 19:04:09.033743 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.033717 2563 manager.go:354] Recovery completed Sep 13 19:04:09.039572 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.039545 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 19:04:09.041654 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.041617 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Sep 13 19:04:09.041739 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.041668 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 19:04:09.041739 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.041679 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Sep 13 19:04:09.042301 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.042285 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 19:04:09.042397 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.042284 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 13 19:04:09.042397 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.042343 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 13 19:04:09.042397 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.042366 2563 state_mem.go:36] "Initialized new in-memory state store" Sep 13 19:04:09.043635 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.043620 2563 policy_none.go:49] "None policy: Start" Sep 13 19:04:09.043710 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.043641 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 13 19:04:09.043710 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.043654 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 13 19:04:09.044712 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.044698 2563 policy_none.go:47] "Start" Sep 13 19:04:09.046379 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.046222 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Sep 13 19:04:09.046446 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.046393 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 19:04:09.046446 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.046408 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Sep 13 19:04:09.054380 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.054287 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-160.ec2.internal.18d4f6be4568f1e4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-160.ec2.internal,UID:ip-10-0-129-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-160.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-160.ec2.internal,},FirstTimestamp:2026-09-13 19:04:09.041654244 +0000 UTC m=+0.423183264,LastTimestamp:2026-09-13 19:04:09.041654244 +0000 UTC m=+0.423183264,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-160.ec2.internal,}" Sep 13 19:04:09.056784 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.056757 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x5z6n" Sep 13 19:04:09.063392 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.063312 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-160.ec2.internal.18d4f6be45693c06 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-160.ec2.internal,UID:ip-10-0-129-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-160.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-160.ec2.internal,},FirstTimestamp:2026-09-13 19:04:09.041673222 +0000 UTC m=+0.423202242,LastTimestamp:2026-09-13 19:04:09.041673222 +0000 UTC m=+0.423202242,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-160.ec2.internal,}" Sep 13 19:04:09.063973 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.063954 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x5z6n" Sep 13 19:04:09.095797 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.095177 2563 manager.go:337] "Starting Device Plugin manager" Sep 13 19:04:09.095797 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.095225 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 13 19:04:09.095797 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.095257 2563 server.go:87] "Starting device plugin registration server" Sep 13 19:04:09.095797 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.095529 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 13 19:04:09.095797 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.095540 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 13 19:04:09.095797 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.095631 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 13 19:04:09.096158 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.095862 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 13 19:04:09.096158 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.095872 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 13 19:04:09.096294 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.096261 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 13 19:04:09.096356 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.096331 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:09.122651 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.122609 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 13 19:04:09.123867 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.123844 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 13 19:04:09.123990 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.123896 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 13 19:04:09.123990 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.123920 2563 kubelet.go:2448] "Starting kubelet main sync loop" Sep 13 19:04:09.124095 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.124005 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 13 19:04:09.125812 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.125784 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 13 19:04:09.196077 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.195974 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 19:04:09.197027 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.197008 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Sep 13 19:04:09.197123 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.197043 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 19:04:09.197123 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.197058 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Sep 13 19:04:09.197123 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.197086 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.204611 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.204590 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.204680 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.204622 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-160.ec2.internal\": node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:09.220276 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.220227 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:09.224496 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.224457 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal"] Sep 13 19:04:09.224567 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.224554 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 19:04:09.225603 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.225584 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Sep 13 19:04:09.225698 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.225620 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 19:04:09.225698 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.225634 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Sep 13 19:04:09.228039 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.228024 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 19:04:09.228170 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.228154 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.228210 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.228189 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 19:04:09.229149 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.229133 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Sep 13 19:04:09.229232 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.229163 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 19:04:09.229232 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.229133 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Sep 13 19:04:09.229232 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.229173 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Sep 13 19:04:09.229232 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.229197 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 19:04:09.229232 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.229209 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Sep 13 19:04:09.231528 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.231512 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.231614 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.231543 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 19:04:09.232490 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.232472 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Sep 13 19:04:09.232596 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.232508 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 19:04:09.232596 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.232522 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Sep 13 19:04:09.258352 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.258323 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-160.ec2.internal\" not found" node="ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.263137 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.263117 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-160.ec2.internal\" not found" node="ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.320864 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.320825 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:09.324152 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.324127 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/82ef5bcb9fdb0c81f8cc0a52c6999929-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal\" (UID: \"82ef5bcb9fdb0c81f8cc0a52c6999929\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.324152 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.324156 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82ef5bcb9fdb0c81f8cc0a52c6999929-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal\" (UID: \"82ef5bcb9fdb0c81f8cc0a52c6999929\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.324348 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.324175 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0df2195e00f1241c11a998a6a4518995-config\") pod \"kube-apiserver-proxy-ip-10-0-129-160.ec2.internal\" (UID: \"0df2195e00f1241c11a998a6a4518995\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.421930 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.421893 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:09.425201 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.425177 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0df2195e00f1241c11a998a6a4518995-config\") pod \"kube-apiserver-proxy-ip-10-0-129-160.ec2.internal\" (UID: \"0df2195e00f1241c11a998a6a4518995\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.425337 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.425213 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/82ef5bcb9fdb0c81f8cc0a52c6999929-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal\" (UID: \"82ef5bcb9fdb0c81f8cc0a52c6999929\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.425337 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.425231 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82ef5bcb9fdb0c81f8cc0a52c6999929-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal\" (UID: \"82ef5bcb9fdb0c81f8cc0a52c6999929\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.425337 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.425310 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/82ef5bcb9fdb0c81f8cc0a52c6999929-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal\" (UID: \"82ef5bcb9fdb0c81f8cc0a52c6999929\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.425437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.425367 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82ef5bcb9fdb0c81f8cc0a52c6999929-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal\" (UID: \"82ef5bcb9fdb0c81f8cc0a52c6999929\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.425437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.425401 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0df2195e00f1241c11a998a6a4518995-config\") pod \"kube-apiserver-proxy-ip-10-0-129-160.ec2.internal\" (UID: \"0df2195e00f1241c11a998a6a4518995\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.522684 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.522653 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:09.560993 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.560959 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.567052 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.565972 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" Sep 13 19:04:09.623497 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.623458 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:09.723884 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.723858 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:09.803752 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.803674 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 13 19:04:09.824593 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.824559 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:09.925098 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:09.925069 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:09.938434 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.938399 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 13 19:04:09.938591 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.938574 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 13 19:04:09.938630 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.938598 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 13 19:04:09.999665 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:09.999630 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 13 19:04:10.025802 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:10.025771 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:10.068203 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.068111 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-12 18:59:09 +0000 UTC" deadline="2028-03-25 15:20:06.786389304 +0000 UTC" Sep 13 19:04:10.068203 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.068154 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13412h15m56.718239698s" Sep 13 19:04:10.125882 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:10.125852 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Sep 13 19:04:10.135030 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.134999 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 19:04:10.136096 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.136070 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Sep 13 19:04:10.136249 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.136114 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 19:04:10.136249 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.136129 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Sep 13 19:04:10.170310 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.170281 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 13 19:04:10.222184 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.222155 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Sep 13 19:04:10.230481 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:10.230443 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df2195e00f1241c11a998a6a4518995.slice/crio-d363bc16d4c9379180d0fcfb54cda12c9f8cdf5f20dacbe3cad0aa6fd997eadd WatchSource:0}: Error finding container d363bc16d4c9379180d0fcfb54cda12c9f8cdf5f20dacbe3cad0aa6fd997eadd: Status 404 returned error can't find the container with id d363bc16d4c9379180d0fcfb54cda12c9f8cdf5f20dacbe3cad0aa6fd997eadd Sep 13 19:04:10.233096 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.233077 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 13 19:04:10.234123 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.234101 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" Sep 13 19:04:10.235343 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.235321 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 13 19:04:10.236382 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:10.236360 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ef5bcb9fdb0c81f8cc0a52c6999929.slice/crio-10f406ccdd326cd63e7ecc989bff54db3ac8a1732ff29a9c779983fe560a9e27 WatchSource:0}: Error finding container 10f406ccdd326cd63e7ecc989bff54db3ac8a1732ff29a9c779983fe560a9e27: Status 404 returned error can't find the container with id 10f406ccdd326cd63e7ecc989bff54db3ac8a1732ff29a9c779983fe560a9e27 Sep 13 19:04:10.241440 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.241417 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 13 19:04:10.981335 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.981303 2563 apiserver.go:52] "Watching apiserver" Sep 13 19:04:10.990324 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.989357 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 13 19:04:10.990324 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.989912 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-nf6g7","kube-system/konnectivity-agent-fflnp","kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal","openshift-dns/node-resolver-ttddh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal","openshift-multus/multus-additional-cni-plugins-fztdh","openshift-multus/multus-lk24l","openshift-multus/network-metrics-daemon-2wc2z","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws","openshift-cluster-node-tuning-operator/tuned-t9kpc","openshift-image-registry/node-ca-fbmxs","openshift-network-diagnostics/network-check-target-525rm","openshift-network-operator/iptables-alerter-m2gng","openshift-ovn-kubernetes/ovnkube-node-tpxtd"] Sep 13 19:04:10.992525 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.992496 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lk24l" Sep 13 19:04:10.995115 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.994712 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fbmxs" Sep 13 19:04:10.995115 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.994749 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 13 19:04:10.995336 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.995202 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fxlq4\"" Sep 13 19:04:10.995336 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.995281 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 13 19:04:10.995478 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.995461 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 13 19:04:10.995714 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.995697 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 13 19:04:10.998906 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.997171 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 13 19:04:10.998906 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.997427 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z8z5h\"" Sep 13 19:04:10.998906 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.997860 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 13 19:04:10.998906 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.998012 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:10.998906 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.998041 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 13 19:04:11.000138 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:10.999879 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ttddh" Sep 13 19:04:11.000795 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.000738 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 13 19:04:11.000795 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.000779 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 13 19:04:11.000970 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.000798 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 13 19:04:11.000970 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.000899 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 13 19:04:11.001094 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.001043 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 13 19:04:11.001322 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.001304 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 13 19:04:11.001429 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.001364 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zbcbs\"" Sep 13 19:04:11.001798 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.001783 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 13 19:04:11.002361 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.002343 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-99jpz\"" Sep 13 19:04:11.002591 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.002574 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 13 19:04:11.002806 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.002789 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:11.002883 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.002860 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:11.006103 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.006078 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.008683 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.008657 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 13 19:04:11.008811 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.008729 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.008892 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.008833 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nj8kn\"" Sep 13 19:04:11.009094 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.009077 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 13 19:04:11.011179 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.011159 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ntxlk\"" Sep 13 19:04:11.011282 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.011267 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m2gng" Sep 13 19:04:11.011491 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.011470 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 13 19:04:11.011614 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.011599 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 13 19:04:11.011684 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.011648 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 13 19:04:11.013742 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.013670 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fflnp" Sep 13 19:04:11.013900 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.013880 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 13 19:04:11.015107 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.014685 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 13 19:04:11.015107 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.015083 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xfcgk\"" Sep 13 19:04:11.015338 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.015316 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 13 19:04:11.016097 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.016079 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 13 19:04:11.017673 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.016304 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 13 19:04:11.017864 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.016429 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nwprg\"" Sep 13 19:04:11.019187 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.019000 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.021593 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.021438 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:11.021593 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.021532 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:11.022335 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.022313 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f2pkm\"" Sep 13 19:04:11.022566 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.022546 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 13 19:04:11.022743 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.022728 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 13 19:04:11.022914 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.022891 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 13 19:04:11.023903 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.023885 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:11.023998 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.023953 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:11.033562 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.033481 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 13 19:04:11.034632 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.034608 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4b7h\" (UniqueName: \"kubernetes.io/projected/044cf94a-295f-4bef-905e-cac6f45bb36b-kube-api-access-x4b7h\") pod \"iptables-alerter-m2gng\" (UID: \"044cf94a-295f-4bef-905e-cac6f45bb36b\") " pod="openshift-network-operator/iptables-alerter-m2gng" Sep 13 19:04:11.034798 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.034778 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-kubernetes\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.034919 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.034904 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-sysctl-d\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.035021 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035008 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-system-cni-dir\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.035120 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035106 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-run-multus-certs\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.035218 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035204 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl86l\" (UniqueName: \"kubernetes.io/projected/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-kube-api-access-jl86l\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:11.035340 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035326 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-modprobe-d\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.035444 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035429 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq4vl\" (UniqueName: \"kubernetes.io/projected/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-kube-api-access-zq4vl\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.035532 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035515 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-multus-socket-dir-parent\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.035600 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035576 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-slash\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.035656 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035610 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd2vc\" (UniqueName: \"kubernetes.io/projected/6b418bcb-8317-400a-adde-d604e722d1b8-kube-api-access-hd2vc\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.035656 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035640 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-dbus\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:11.035770 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035664 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-etc-selinux\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.035770 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035687 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-etc-kubernetes\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.035770 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035709 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-kubelet\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.035770 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035736 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-run\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.035770 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035758 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-systemd-units\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.036006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035778 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-host\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.036006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035820 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc7qx\" (UniqueName: \"kubernetes.io/projected/80f433a2-abd8-460f-848a-91dda050f334-kube-api-access-nc7qx\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.036006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035850 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:11.036006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035876 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6b418bcb-8317-400a-adde-d604e722d1b8-ovn-node-metrics-cert\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.036006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035892 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31fa23e6-3a87-4f8f-9f54-48853123eb54-hosts-file\") pod \"node-resolver-ttddh\" (UID: \"31fa23e6-3a87-4f8f-9f54-48853123eb54\") " pod="openshift-dns/node-resolver-ttddh" Sep 13 19:04:11.036006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035919 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.036006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035942 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56671f6f-f921-4342-a427-bb5d123e57a9-cni-binary-copy\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.036006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035963 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-run-netns\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.036006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.035978 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80f433a2-abd8-460f-848a-91dda050f334-etc-tuned\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036006 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-sys-fs\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036044 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-multus-conf-dir\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036075 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/effec463-bf64-4a1c-b7ad-9bee7e4901df-serviceca\") pod \"node-ca-fbmxs\" (UID: \"effec463-bf64-4a1c-b7ad-9bee7e4901df\") " pod="openshift-image-registry/node-ca-fbmxs" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036099 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036138 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80f433a2-abd8-460f-848a-91dda050f334-tmp\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-system-cni-dir\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036220 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-cnibin\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036277 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6b418bcb-8317-400a-adde-d604e722d1b8-env-overrides\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036305 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/044cf94a-295f-4bef-905e-cac6f45bb36b-iptables-alerter-script\") pod \"iptables-alerter-m2gng\" (UID: \"044cf94a-295f-4bef-905e-cac6f45bb36b\") " pod="openshift-network-operator/iptables-alerter-m2gng" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036327 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-var-lib-kubelet\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036351 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-cni-binary-copy\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036373 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-run-systemd\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036394 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-cni-bin\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036417 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-sysconfig\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.036437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036440 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-sysctl-conf\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036463 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-hostroot\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036485 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-var-lib-openvswitch\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036508 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-node-log\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036530 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-log-socket\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036555 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036579 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036607 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-run-k8s-cni-cncf-io\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036630 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-var-lib-cni-bin\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036654 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-kubelet-config\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036678 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-registration-dir\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036732 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzsw9\" (UniqueName: \"kubernetes.io/projected/45a03def-1925-4d5b-b8fe-b1449cce31fe-kube-api-access-jzsw9\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036760 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-run-netns\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036827 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-var-lib-cni-multus\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036930 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-var-lib-kubelet\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036955 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh5xs\" (UniqueName: \"kubernetes.io/projected/56671f6f-f921-4342-a427-bb5d123e57a9-kube-api-access-fh5xs\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.037136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.036976 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-etc-openvswitch\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037028 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-sys\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037063 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-cnibin\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037100 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037127 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-multus-cni-dir\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037204 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037292 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-cni-netd\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037319 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6b418bcb-8317-400a-adde-d604e722d1b8-ovnkube-script-lib\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037346 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6b87\" (UniqueName: \"kubernetes.io/projected/31fa23e6-3a87-4f8f-9f54-48853123eb54-kube-api-access-x6b87\") pod \"node-resolver-ttddh\" (UID: \"31fa23e6-3a87-4f8f-9f54-48853123eb54\") " pod="openshift-dns/node-resolver-ttddh" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-socket-dir\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037393 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8738a08c-3f02-4fa0-b77c-51357c3eb253-agent-certs\") pod \"konnectivity-agent-fflnp\" (UID: \"8738a08c-3f02-4fa0-b77c-51357c3eb253\") " pod="kube-system/konnectivity-agent-fflnp" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037418 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8738a08c-3f02-4fa0-b77c-51357c3eb253-konnectivity-ca\") pod \"konnectivity-agent-fflnp\" (UID: \"8738a08c-3f02-4fa0-b77c-51357c3eb253\") " pod="kube-system/konnectivity-agent-fflnp" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037442 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56671f6f-f921-4342-a427-bb5d123e57a9-multus-daemon-config\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037466 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037489 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/044cf94a-295f-4bef-905e-cac6f45bb36b-host-slash\") pod \"iptables-alerter-m2gng\" (UID: \"044cf94a-295f-4bef-905e-cac6f45bb36b\") " pod="openshift-network-operator/iptables-alerter-m2gng" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037511 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-systemd\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.038004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037535 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrbhx\" (UniqueName: \"kubernetes.io/projected/effec463-bf64-4a1c-b7ad-9bee7e4901df-kube-api-access-vrbhx\") pod \"node-ca-fbmxs\" (UID: \"effec463-bf64-4a1c-b7ad-9bee7e4901df\") " pod="openshift-image-registry/node-ca-fbmxs" Sep 13 19:04:11.038745 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037567 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-run-ovn\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.038745 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037597 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6b418bcb-8317-400a-adde-d604e722d1b8-ovnkube-config\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.038745 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037632 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31fa23e6-3a87-4f8f-9f54-48853123eb54-tmp-dir\") pod \"node-resolver-ttddh\" (UID: \"31fa23e6-3a87-4f8f-9f54-48853123eb54\") " pod="openshift-dns/node-resolver-ttddh" Sep 13 19:04:11.038745 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037654 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-lib-modules\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.038745 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037675 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-device-dir\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.038745 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037714 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fgfl\" (UniqueName: \"kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl\") pod \"network-check-target-525rm\" (UID: \"e9f0b4f6-e276-412b-9055-0389f82519db\") " pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:11.038745 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037755 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/effec463-bf64-4a1c-b7ad-9bee7e4901df-host\") pod \"node-ca-fbmxs\" (UID: \"effec463-bf64-4a1c-b7ad-9bee7e4901df\") " pod="openshift-image-registry/node-ca-fbmxs" Sep 13 19:04:11.038745 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.037784 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-run-openvswitch\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.050373 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.050324 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mppv6" Sep 13 19:04:11.057821 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.057784 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mppv6" Sep 13 19:04:11.123951 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.123917 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 13 19:04:11.129375 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.129309 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" event={"ID":"82ef5bcb9fdb0c81f8cc0a52c6999929","Type":"ContainerStarted","Data":"10f406ccdd326cd63e7ecc989bff54db3ac8a1732ff29a9c779983fe560a9e27"} Sep 13 19:04:11.131483 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.131450 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" event={"ID":"0df2195e00f1241c11a998a6a4518995","Type":"ContainerStarted","Data":"d363bc16d4c9379180d0fcfb54cda12c9f8cdf5f20dacbe3cad0aa6fd997eadd"} Sep 13 19:04:11.139099 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139044 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/044cf94a-295f-4bef-905e-cac6f45bb36b-host-slash\") pod \"iptables-alerter-m2gng\" (UID: \"044cf94a-295f-4bef-905e-cac6f45bb36b\") " pod="openshift-network-operator/iptables-alerter-m2gng" Sep 13 19:04:11.139327 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139120 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-systemd\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.139327 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139155 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrbhx\" (UniqueName: \"kubernetes.io/projected/effec463-bf64-4a1c-b7ad-9bee7e4901df-kube-api-access-vrbhx\") pod \"node-ca-fbmxs\" (UID: \"effec463-bf64-4a1c-b7ad-9bee7e4901df\") " pod="openshift-image-registry/node-ca-fbmxs" Sep 13 19:04:11.139327 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139190 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-run-ovn\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.139327 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139218 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6b418bcb-8317-400a-adde-d604e722d1b8-ovnkube-config\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.139327 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139265 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31fa23e6-3a87-4f8f-9f54-48853123eb54-tmp-dir\") pod \"node-resolver-ttddh\" (UID: \"31fa23e6-3a87-4f8f-9f54-48853123eb54\") " pod="openshift-dns/node-resolver-ttddh" Sep 13 19:04:11.139327 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139300 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-lib-modules\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.139599 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139331 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-device-dir\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.139599 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139369 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fgfl\" (UniqueName: \"kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl\") pod \"network-check-target-525rm\" (UID: \"e9f0b4f6-e276-412b-9055-0389f82519db\") " pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:11.139599 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139396 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/effec463-bf64-4a1c-b7ad-9bee7e4901df-host\") pod \"node-ca-fbmxs\" (UID: \"effec463-bf64-4a1c-b7ad-9bee7e4901df\") " pod="openshift-image-registry/node-ca-fbmxs" Sep 13 19:04:11.139599 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139426 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-run-openvswitch\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.139599 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139458 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4b7h\" (UniqueName: \"kubernetes.io/projected/044cf94a-295f-4bef-905e-cac6f45bb36b-kube-api-access-x4b7h\") pod \"iptables-alerter-m2gng\" (UID: \"044cf94a-295f-4bef-905e-cac6f45bb36b\") " pod="openshift-network-operator/iptables-alerter-m2gng" Sep 13 19:04:11.139814 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139631 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-kubernetes\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.139814 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139703 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/044cf94a-295f-4bef-905e-cac6f45bb36b-host-slash\") pod \"iptables-alerter-m2gng\" (UID: \"044cf94a-295f-4bef-905e-cac6f45bb36b\") " pod="openshift-network-operator/iptables-alerter-m2gng" Sep 13 19:04:11.139814 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139755 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-systemd\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.139943 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139823 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-lib-modules\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.139943 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139861 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-device-dir\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.139943 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139917 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/effec463-bf64-4a1c-b7ad-9bee7e4901df-host\") pod \"node-ca-fbmxs\" (UID: \"effec463-bf64-4a1c-b7ad-9bee7e4901df\") " pod="openshift-image-registry/node-ca-fbmxs" Sep 13 19:04:11.140115 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.139964 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-run-openvswitch\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.140163 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.140132 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-run-ovn\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.140612 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.140577 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31fa23e6-3a87-4f8f-9f54-48853123eb54-tmp-dir\") pod \"node-resolver-ttddh\" (UID: \"31fa23e6-3a87-4f8f-9f54-48853123eb54\") " pod="openshift-dns/node-resolver-ttddh" Sep 13 19:04:11.140908 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.140886 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6b418bcb-8317-400a-adde-d604e722d1b8-ovnkube-config\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.141028 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.140968 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-kubernetes\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.141076 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141060 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-sysctl-d\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.141130 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141095 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-system-cni-dir\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.141130 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-run-multus-certs\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.141221 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141149 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl86l\" (UniqueName: \"kubernetes.io/projected/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-kube-api-access-jl86l\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:11.141221 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141179 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-modprobe-d\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.141221 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141208 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq4vl\" (UniqueName: \"kubernetes.io/projected/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-kube-api-access-zq4vl\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.141375 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141254 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-multus-socket-dir-parent\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.141375 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141280 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-slash\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.141375 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141311 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd2vc\" (UniqueName: \"kubernetes.io/projected/6b418bcb-8317-400a-adde-d604e722d1b8-kube-api-access-hd2vc\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.141375 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141341 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-dbus\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:11.141536 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141378 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-etc-selinux\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.141536 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141405 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-etc-kubernetes\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.141536 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141430 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-kubelet\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.141536 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141459 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-run\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.141536 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141489 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-systemd-units\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.141536 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141514 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-host\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.141779 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141544 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc7qx\" (UniqueName: \"kubernetes.io/projected/80f433a2-abd8-460f-848a-91dda050f334-kube-api-access-nc7qx\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.141779 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141573 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:11.141779 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141596 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6b418bcb-8317-400a-adde-d604e722d1b8-ovn-node-metrics-cert\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.141779 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141634 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31fa23e6-3a87-4f8f-9f54-48853123eb54-hosts-file\") pod \"node-resolver-ttddh\" (UID: \"31fa23e6-3a87-4f8f-9f54-48853123eb54\") " pod="openshift-dns/node-resolver-ttddh" Sep 13 19:04:11.141779 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141664 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.141779 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141693 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56671f6f-f921-4342-a427-bb5d123e57a9-cni-binary-copy\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.141779 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141717 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-run-netns\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.141779 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80f433a2-abd8-460f-848a-91dda050f334-etc-tuned\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.141779 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141771 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-sys-fs\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141800 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-multus-conf-dir\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141829 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/effec463-bf64-4a1c-b7ad-9bee7e4901df-serviceca\") pod \"node-ca-fbmxs\" (UID: \"effec463-bf64-4a1c-b7ad-9bee7e4901df\") " pod="openshift-image-registry/node-ca-fbmxs" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141853 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141880 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80f433a2-abd8-460f-848a-91dda050f334-tmp\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141907 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-system-cni-dir\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141937 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-cnibin\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141968 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6b418bcb-8317-400a-adde-d604e722d1b8-env-overrides\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.141999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/044cf94a-295f-4bef-905e-cac6f45bb36b-iptables-alerter-script\") pod \"iptables-alerter-m2gng\" (UID: \"044cf94a-295f-4bef-905e-cac6f45bb36b\") " pod="openshift-network-operator/iptables-alerter-m2gng" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142030 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-var-lib-kubelet\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142057 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-cni-binary-copy\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142090 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-run-systemd\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142118 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-cni-bin\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142148 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-sysconfig\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.142173 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142174 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-sysctl-conf\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142202 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-hostroot\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142232 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-var-lib-openvswitch\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142280 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-node-log\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142309 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-log-socket\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142334 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142367 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142401 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-run-k8s-cni-cncf-io\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142432 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-var-lib-cni-bin\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142463 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-kubelet-config\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142489 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-registration-dir\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142522 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzsw9\" (UniqueName: \"kubernetes.io/projected/45a03def-1925-4d5b-b8fe-b1449cce31fe-kube-api-access-jzsw9\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142552 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-run-netns\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142584 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-var-lib-cni-multus\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142611 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-var-lib-kubelet\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142641 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh5xs\" (UniqueName: \"kubernetes.io/projected/56671f6f-f921-4342-a427-bb5d123e57a9-kube-api-access-fh5xs\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142671 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-etc-openvswitch\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.142839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142702 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-sys\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142731 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-cnibin\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142766 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142796 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-multus-cni-dir\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142856 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-cni-netd\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142886 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6b418bcb-8317-400a-adde-d604e722d1b8-ovnkube-script-lib\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142913 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6b87\" (UniqueName: \"kubernetes.io/projected/31fa23e6-3a87-4f8f-9f54-48853123eb54-kube-api-access-x6b87\") pod \"node-resolver-ttddh\" (UID: \"31fa23e6-3a87-4f8f-9f54-48853123eb54\") " pod="openshift-dns/node-resolver-ttddh" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142944 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-socket-dir\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.142975 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8738a08c-3f02-4fa0-b77c-51357c3eb253-agent-certs\") pod \"konnectivity-agent-fflnp\" (UID: \"8738a08c-3f02-4fa0-b77c-51357c3eb253\") " pod="kube-system/konnectivity-agent-fflnp" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.143019 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8738a08c-3f02-4fa0-b77c-51357c3eb253-konnectivity-ca\") pod \"konnectivity-agent-fflnp\" (UID: \"8738a08c-3f02-4fa0-b77c-51357c3eb253\") " pod="kube-system/konnectivity-agent-fflnp" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.143047 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56671f6f-f921-4342-a427-bb5d123e57a9-multus-daemon-config\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.143079 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.143574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.143506 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-var-lib-kubelet\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.144263 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.143740 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-sysctl-d\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.144263 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.143811 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-etc-openvswitch\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.144263 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.143872 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.144263 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.143934 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-cnibin\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.144263 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.143998 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-multus-conf-dir\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.144263 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144074 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.144263 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144170 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-cnibin\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.144263 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144225 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-system-cni-dir\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.144647 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144293 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-run-multus-certs\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.144647 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144291 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31fa23e6-3a87-4f8f-9f54-48853123eb54-hosts-file\") pod \"node-resolver-ttddh\" (UID: \"31fa23e6-3a87-4f8f-9f54-48853123eb54\") " pod="openshift-dns/node-resolver-ttddh" Sep 13 19:04:11.144647 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144335 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-systemd-units\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.144647 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144416 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-var-lib-cni-multus\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.144647 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144480 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-multus-socket-dir-parent\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.144647 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144526 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-slash\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.144647 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144534 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 13 19:04:11.144647 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144640 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-sysconfig\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.145019 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144663 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-run-systemd\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.145019 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144670 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-cni-bin\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.145019 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.144759 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:11.145019 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.144832 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs podName:526ccac7-e62a-45c1-bc16-1c2f1bc7aca5 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:11.644811818 +0000 UTC m=+3.026340847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs") pod "network-metrics-daemon-2wc2z" (UID: "526ccac7-e62a-45c1-bc16-1c2f1bc7aca5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:11.145019 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.144912 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-cni-binary-copy\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.145291 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.145083 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-hostroot\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.145291 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.145142 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-var-lib-openvswitch\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.145291 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.145196 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-node-log\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.145291 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.145197 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6b418bcb-8317-400a-adde-d604e722d1b8-ovnkube-script-lib\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.145491 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.145294 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6b418bcb-8317-400a-adde-d604e722d1b8-env-overrides\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.145491 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.145299 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-dbus\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:11.145491 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.145334 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-log-socket\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.145491 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.145448 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-run-netns\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.145667 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.145520 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-run-k8s-cni-cncf-io\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.145667 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.145608 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-sys-fs\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.145786 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.145766 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:11.145873 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.145853 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret podName:af6ceb0a-0f1f-42f9-8328-dc6263f46fe2 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:11.645839392 +0000 UTC m=+3.027368404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret") pod "global-pull-secret-syncer-nf6g7" (UID: "af6ceb0a-0f1f-42f9-8328-dc6263f46fe2") : object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:11.145952 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.145933 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-cni-netd\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.146383 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.146350 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-run\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.146485 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.146391 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-modprobe-d\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.146485 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.146410 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56671f6f-f921-4342-a427-bb5d123e57a9-multus-daemon-config\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.146595 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.146512 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.146595 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.146529 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-etc-sysctl-conf\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.146595 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.146583 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-var-lib-kubelet\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.146783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.146643 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-etc-selinux\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.146783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.146690 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-etc-kubernetes\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.146783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.146732 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b418bcb-8317-400a-adde-d604e722d1b8-host-kubelet\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.146914 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.146797 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-multus-cni-dir\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.146914 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.146849 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-host\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.147339 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.147314 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-kubelet-config\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:11.147425 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.147406 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-registration-dir\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.147487 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.147474 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.147617 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.147594 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/044cf94a-295f-4bef-905e-cac6f45bb36b-iptables-alerter-script\") pod \"iptables-alerter-m2gng\" (UID: \"044cf94a-295f-4bef-905e-cac6f45bb36b\") " pod="openshift-network-operator/iptables-alerter-m2gng" Sep 13 19:04:11.147705 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.147680 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-var-lib-cni-bin\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.147833 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.147808 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-host-run-netns\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.147912 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.147871 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80f433a2-abd8-460f-848a-91dda050f334-sys\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.148419 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.148037 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.148419 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.147811 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/effec463-bf64-4a1c-b7ad-9bee7e4901df-serviceca\") pod \"node-ca-fbmxs\" (UID: \"effec463-bf64-4a1c-b7ad-9bee7e4901df\") " pod="openshift-image-registry/node-ca-fbmxs" Sep 13 19:04:11.148419 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.148366 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 19:04:11.148419 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.148388 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 19:04:11.148419 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.148402 2563 projected.go:196] Error preparing data for projected volume kube-api-access-7fgfl for pod openshift-network-diagnostics/network-check-target-525rm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:11.148694 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.148449 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56671f6f-f921-4342-a427-bb5d123e57a9-system-cni-dir\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.148694 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.148498 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl podName:e9f0b4f6-e276-412b-9055-0389f82519db nodeName:}" failed. No retries permitted until 2026-09-13 19:04:11.648470741 +0000 UTC m=+3.029999769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7fgfl" (UniqueName: "kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl") pod "network-check-target-525rm" (UID: "e9f0b4f6-e276-412b-9055-0389f82519db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:11.148694 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.148595 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56671f6f-f921-4342-a427-bb5d123e57a9-cni-binary-copy\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.148855 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.148801 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.149031 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.149012 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/45a03def-1925-4d5b-b8fe-b1449cce31fe-socket-dir\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.149270 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.148979 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8738a08c-3f02-4fa0-b77c-51357c3eb253-konnectivity-ca\") pod \"konnectivity-agent-fflnp\" (UID: \"8738a08c-3f02-4fa0-b77c-51357c3eb253\") " pod="kube-system/konnectivity-agent-fflnp" Sep 13 19:04:11.152444 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.150608 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80f433a2-abd8-460f-848a-91dda050f334-etc-tuned\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.153264 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.152872 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc7qx\" (UniqueName: \"kubernetes.io/projected/80f433a2-abd8-460f-848a-91dda050f334-kube-api-access-nc7qx\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.153264 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.152893 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh5xs\" (UniqueName: \"kubernetes.io/projected/56671f6f-f921-4342-a427-bb5d123e57a9-kube-api-access-fh5xs\") pod \"multus-lk24l\" (UID: \"56671f6f-f921-4342-a427-bb5d123e57a9\") " pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.153264 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.152891 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4b7h\" (UniqueName: \"kubernetes.io/projected/044cf94a-295f-4bef-905e-cac6f45bb36b-kube-api-access-x4b7h\") pod \"iptables-alerter-m2gng\" (UID: \"044cf94a-295f-4bef-905e-cac6f45bb36b\") " pod="openshift-network-operator/iptables-alerter-m2gng" Sep 13 19:04:11.153264 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.152872 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrbhx\" (UniqueName: \"kubernetes.io/projected/effec463-bf64-4a1c-b7ad-9bee7e4901df-kube-api-access-vrbhx\") pod \"node-ca-fbmxs\" (UID: \"effec463-bf64-4a1c-b7ad-9bee7e4901df\") " pod="openshift-image-registry/node-ca-fbmxs" Sep 13 19:04:11.153264 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.153131 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6b418bcb-8317-400a-adde-d604e722d1b8-ovn-node-metrics-cert\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.153688 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.153276 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8738a08c-3f02-4fa0-b77c-51357c3eb253-agent-certs\") pod \"konnectivity-agent-fflnp\" (UID: \"8738a08c-3f02-4fa0-b77c-51357c3eb253\") " pod="kube-system/konnectivity-agent-fflnp" Sep 13 19:04:11.153921 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.153895 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80f433a2-abd8-460f-848a-91dda050f334-tmp\") pod \"tuned-t9kpc\" (UID: \"80f433a2-abd8-460f-848a-91dda050f334\") " pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.155635 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.155609 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl86l\" (UniqueName: \"kubernetes.io/projected/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-kube-api-access-jl86l\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:11.156070 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.156048 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq4vl\" (UniqueName: \"kubernetes.io/projected/60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c-kube-api-access-zq4vl\") pod \"multus-additional-cni-plugins-fztdh\" (UID: \"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c\") " pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.156483 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.156412 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd2vc\" (UniqueName: \"kubernetes.io/projected/6b418bcb-8317-400a-adde-d604e722d1b8-kube-api-access-hd2vc\") pod \"ovnkube-node-tpxtd\" (UID: \"6b418bcb-8317-400a-adde-d604e722d1b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.157523 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.157495 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6b87\" (UniqueName: \"kubernetes.io/projected/31fa23e6-3a87-4f8f-9f54-48853123eb54-kube-api-access-x6b87\") pod \"node-resolver-ttddh\" (UID: \"31fa23e6-3a87-4f8f-9f54-48853123eb54\") " pod="openshift-dns/node-resolver-ttddh" Sep 13 19:04:11.158832 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.158809 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzsw9\" (UniqueName: \"kubernetes.io/projected/45a03def-1925-4d5b-b8fe-b1449cce31fe-kube-api-access-jzsw9\") pod \"aws-ebs-csi-driver-node-2gqws\" (UID: \"45a03def-1925-4d5b-b8fe-b1449cce31fe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.213880 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.213846 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 13 19:04:11.307662 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.307568 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lk24l" Sep 13 19:04:11.317608 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.317572 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:11.326520 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.326484 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fbmxs" Sep 13 19:04:11.333653 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.333616 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ttddh" Sep 13 19:04:11.340869 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.340835 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fztdh" Sep 13 19:04:11.348481 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.348446 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" Sep 13 19:04:11.357451 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.357419 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m2gng" Sep 13 19:04:11.363498 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.363469 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fflnp" Sep 13 19:04:11.369657 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.369625 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" Sep 13 19:04:11.647919 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.647822 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:11.647919 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.647871 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:11.648176 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.647996 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:11.648176 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.648025 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:11.648176 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.648070 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret podName:af6ceb0a-0f1f-42f9-8328-dc6263f46fe2 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:12.648053581 +0000 UTC m=+4.029582589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret") pod "global-pull-secret-syncer-nf6g7" (UID: "af6ceb0a-0f1f-42f9-8328-dc6263f46fe2") : object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:11.648176 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.648087 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs podName:526ccac7-e62a-45c1-bc16-1c2f1bc7aca5 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:12.648079756 +0000 UTC m=+4.029608764 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs") pod "network-metrics-daemon-2wc2z" (UID: "526ccac7-e62a-45c1-bc16-1c2f1bc7aca5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:11.748761 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.748717 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fgfl\" (UniqueName: \"kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl\") pod \"network-check-target-525rm\" (UID: \"e9f0b4f6-e276-412b-9055-0389f82519db\") " pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:11.748952 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.748852 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 19:04:11.748952 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.748876 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 19:04:11.748952 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.748888 2563 projected.go:196] Error preparing data for projected volume kube-api-access-7fgfl for pod openshift-network-diagnostics/network-check-target-525rm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:11.749139 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:11.748950 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl podName:e9f0b4f6-e276-412b-9055-0389f82519db nodeName:}" failed. No retries permitted until 2026-09-13 19:04:12.748929232 +0000 UTC m=+4.130458259 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7fgfl" (UniqueName: "kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl") pod "network-check-target-525rm" (UID: "e9f0b4f6-e276-412b-9055-0389f82519db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:11.751776 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:11.751748 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 13 19:04:12.062021 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:12.061960 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-12 18:59:11 +0000 UTC" deadline="2028-02-22 07:05:25.103971817 +0000 UTC" Sep 13 19:04:12.062021 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:12.062015 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12636h1m13.041962264s" Sep 13 19:04:12.124631 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:12.124591 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:12.124804 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:12.124710 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:12.189007 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:12.188975 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f433a2_abd8_460f_848a_91dda050f334.slice/crio-eb8e2aa466d8b5b99dd6835ab395e41d13dae1b84c5eba8ba1f3e3506ff0289c WatchSource:0}: Error finding container eb8e2aa466d8b5b99dd6835ab395e41d13dae1b84c5eba8ba1f3e3506ff0289c: Status 404 returned error can't find the container with id eb8e2aa466d8b5b99dd6835ab395e41d13dae1b84c5eba8ba1f3e3506ff0289c Sep 13 19:04:12.190500 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:12.190472 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeffec463_bf64_4a1c_b7ad_9bee7e4901df.slice/crio-8ee51bac507c1b73d7fb200574f325a9d09b4fcce3e6957975dbf8af880e75da WatchSource:0}: Error finding container 8ee51bac507c1b73d7fb200574f325a9d09b4fcce3e6957975dbf8af880e75da: Status 404 returned error can't find the container with id 8ee51bac507c1b73d7fb200574f325a9d09b4fcce3e6957975dbf8af880e75da Sep 13 19:04:12.193972 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:12.193555 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b418bcb_8317_400a_adde_d604e722d1b8.slice/crio-6b8a71fda68f4021bb345c16073703161e6b9cbb0ef2fedf5cc1a5b01c05aa8d WatchSource:0}: Error finding container 6b8a71fda68f4021bb345c16073703161e6b9cbb0ef2fedf5cc1a5b01c05aa8d: Status 404 returned error can't find the container with id 6b8a71fda68f4021bb345c16073703161e6b9cbb0ef2fedf5cc1a5b01c05aa8d Sep 13 19:04:12.196984 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:12.196960 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod044cf94a_295f_4bef_905e_cac6f45bb36b.slice/crio-b0411f5f729545366def1938b2569d53ce7b5cca7b9c281725f70e702ccf2a0b WatchSource:0}: Error finding container b0411f5f729545366def1938b2569d53ce7b5cca7b9c281725f70e702ccf2a0b: Status 404 returned error can't find the container with id b0411f5f729545366def1938b2569d53ce7b5cca7b9c281725f70e702ccf2a0b Sep 13 19:04:12.198572 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:12.198460 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56671f6f_f921_4342_a427_bb5d123e57a9.slice/crio-45edd460ce47429539ad85040b360e351888e9c4c792d71ae1adfe1d24daa24e WatchSource:0}: Error finding container 45edd460ce47429539ad85040b360e351888e9c4c792d71ae1adfe1d24daa24e: Status 404 returned error can't find the container with id 45edd460ce47429539ad85040b360e351888e9c4c792d71ae1adfe1d24daa24e Sep 13 19:04:12.199965 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:12.199923 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e1f7fc_ed12_4ed7_bb9c_bcceeca0533c.slice/crio-5e871a923f1dc0fc88cc0e8846d1678135a42537e1c83a9abf66f0d6b85f5bad WatchSource:0}: Error finding container 5e871a923f1dc0fc88cc0e8846d1678135a42537e1c83a9abf66f0d6b85f5bad: Status 404 returned error can't find the container with id 5e871a923f1dc0fc88cc0e8846d1678135a42537e1c83a9abf66f0d6b85f5bad Sep 13 19:04:12.200913 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:12.200679 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8738a08c_3f02_4fa0_b77c_51357c3eb253.slice/crio-8d115ed4c1cf346b99cb086376eb79ba40714c48595ae6151393054ee4e3ed2b WatchSource:0}: Error finding container 8d115ed4c1cf346b99cb086376eb79ba40714c48595ae6151393054ee4e3ed2b: Status 404 returned error can't find the container with id 8d115ed4c1cf346b99cb086376eb79ba40714c48595ae6151393054ee4e3ed2b Sep 13 19:04:12.201874 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:12.201847 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45a03def_1925_4d5b_b8fe_b1449cce31fe.slice/crio-6d925a56886d78549521ee79a2e9d93b41cf59c07034f86da438ac88edb7d6a0 WatchSource:0}: Error finding container 6d925a56886d78549521ee79a2e9d93b41cf59c07034f86da438ac88edb7d6a0: Status 404 returned error can't find the container with id 6d925a56886d78549521ee79a2e9d93b41cf59c07034f86da438ac88edb7d6a0 Sep 13 19:04:12.203200 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:12.203118 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31fa23e6_3a87_4f8f_9f54_48853123eb54.slice/crio-47dbd6b58eac0d5cb7f2d0c4de5cb244ed18924b9c2e9eb5a9654af4b87e4782 WatchSource:0}: Error finding container 47dbd6b58eac0d5cb7f2d0c4de5cb244ed18924b9c2e9eb5a9654af4b87e4782: Status 404 returned error can't find the container with id 47dbd6b58eac0d5cb7f2d0c4de5cb244ed18924b9c2e9eb5a9654af4b87e4782 Sep 13 19:04:12.657263 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:12.656986 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:12.657467 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:12.657278 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:12.657467 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:12.657192 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:12.657467 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:12.657403 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret podName:af6ceb0a-0f1f-42f9-8328-dc6263f46fe2 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:14.657379832 +0000 UTC m=+6.038908856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret") pod "global-pull-secret-syncer-nf6g7" (UID: "af6ceb0a-0f1f-42f9-8328-dc6263f46fe2") : object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:12.657467 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:12.657442 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:12.657709 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:12.657505 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs podName:526ccac7-e62a-45c1-bc16-1c2f1bc7aca5 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:14.657487145 +0000 UTC m=+6.039016172 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs") pod "network-metrics-daemon-2wc2z" (UID: "526ccac7-e62a-45c1-bc16-1c2f1bc7aca5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:12.758638 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:12.758468 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fgfl\" (UniqueName: \"kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl\") pod \"network-check-target-525rm\" (UID: \"e9f0b4f6-e276-412b-9055-0389f82519db\") " pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:12.758916 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:12.758672 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 19:04:12.758916 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:12.758698 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 19:04:12.758916 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:12.758712 2563 projected.go:196] Error preparing data for projected volume kube-api-access-7fgfl for pod openshift-network-diagnostics/network-check-target-525rm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:12.758916 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:12.758775 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl podName:e9f0b4f6-e276-412b-9055-0389f82519db nodeName:}" failed. No retries permitted until 2026-09-13 19:04:14.758755841 +0000 UTC m=+6.140284857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7fgfl" (UniqueName: "kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl") pod "network-check-target-525rm" (UID: "e9f0b4f6-e276-412b-9055-0389f82519db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:13.062483 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.062359 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-12 18:59:11 +0000 UTC" deadline="2028-05-29 13:42:07.213685842 +0000 UTC" Sep 13 19:04:13.062483 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.062418 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14970h37m54.151273352s" Sep 13 19:04:13.127281 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.127239 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:13.127509 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:13.127439 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:13.127887 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.127865 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:13.127997 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:13.127977 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:13.146417 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.146342 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" event={"ID":"6b418bcb-8317-400a-adde-d604e722d1b8","Type":"ContainerStarted","Data":"6b8a71fda68f4021bb345c16073703161e6b9cbb0ef2fedf5cc1a5b01c05aa8d"} Sep 13 19:04:13.155966 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.155909 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fbmxs" event={"ID":"effec463-bf64-4a1c-b7ad-9bee7e4901df","Type":"ContainerStarted","Data":"8ee51bac507c1b73d7fb200574f325a9d09b4fcce3e6957975dbf8af880e75da"} Sep 13 19:04:13.174346 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.173742 2563 generic.go:356] "Generic (PLEG): container finished" podID="82ef5bcb9fdb0c81f8cc0a52c6999929" containerID="e808152558ea46f91a4a31c4c9d4cfcd5b7ed63a58dc20a7726e45328b2b993b" exitCode=0 Sep 13 19:04:13.174346 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.173876 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" event={"ID":"82ef5bcb9fdb0c81f8cc0a52c6999929","Type":"ContainerDied","Data":"e808152558ea46f91a4a31c4c9d4cfcd5b7ed63a58dc20a7726e45328b2b993b"} Sep 13 19:04:13.196140 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.196095 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" event={"ID":"0df2195e00f1241c11a998a6a4518995","Type":"ContainerStarted","Data":"4659074b701345658999c557e27bc4edd65e064395579e939b141a86917a97a7"} Sep 13 19:04:13.212992 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.212923 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" podStartSLOduration=3.212905562 podStartE2EDuration="3.212905562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:04:13.211981839 +0000 UTC m=+4.593510864" watchObservedRunningTime="2026-09-13 19:04:13.212905562 +0000 UTC m=+4.594434594" Sep 13 19:04:13.213211 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.213185 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ttddh" event={"ID":"31fa23e6-3a87-4f8f-9f54-48853123eb54","Type":"ContainerStarted","Data":"47dbd6b58eac0d5cb7f2d0c4de5cb244ed18924b9c2e9eb5a9654af4b87e4782"} Sep 13 19:04:13.217337 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.217180 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" event={"ID":"80f433a2-abd8-460f-848a-91dda050f334","Type":"ContainerStarted","Data":"eb8e2aa466d8b5b99dd6835ab395e41d13dae1b84c5eba8ba1f3e3506ff0289c"} Sep 13 19:04:13.229407 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.229367 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" event={"ID":"45a03def-1925-4d5b-b8fe-b1449cce31fe","Type":"ContainerStarted","Data":"6d925a56886d78549521ee79a2e9d93b41cf59c07034f86da438ac88edb7d6a0"} Sep 13 19:04:13.240197 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.240158 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fflnp" event={"ID":"8738a08c-3f02-4fa0-b77c-51357c3eb253","Type":"ContainerStarted","Data":"8d115ed4c1cf346b99cb086376eb79ba40714c48595ae6151393054ee4e3ed2b"} Sep 13 19:04:13.246275 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.246141 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fztdh" event={"ID":"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c","Type":"ContainerStarted","Data":"5e871a923f1dc0fc88cc0e8846d1678135a42537e1c83a9abf66f0d6b85f5bad"} Sep 13 19:04:13.255382 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.255212 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lk24l" event={"ID":"56671f6f-f921-4342-a427-bb5d123e57a9","Type":"ContainerStarted","Data":"45edd460ce47429539ad85040b360e351888e9c4c792d71ae1adfe1d24daa24e"} Sep 13 19:04:13.261254 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:13.261211 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m2gng" event={"ID":"044cf94a-295f-4bef-905e-cac6f45bb36b","Type":"ContainerStarted","Data":"b0411f5f729545366def1938b2569d53ce7b5cca7b9c281725f70e702ccf2a0b"} Sep 13 19:04:14.125233 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:14.125194 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:14.125889 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:14.125351 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:14.286920 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:14.286734 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" event={"ID":"82ef5bcb9fdb0c81f8cc0a52c6999929","Type":"ContainerStarted","Data":"494039f3daceee0b6cc89b31c6e2a83bc9d30ebaa001c33f14b9524b31fe0def"} Sep 13 19:04:14.679986 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:14.679019 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:14.679986 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:14.679086 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:14.679986 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:14.679329 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:14.679986 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:14.679405 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs podName:526ccac7-e62a-45c1-bc16-1c2f1bc7aca5 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:18.679383998 +0000 UTC m=+10.060913031 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs") pod "network-metrics-daemon-2wc2z" (UID: "526ccac7-e62a-45c1-bc16-1c2f1bc7aca5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:14.679986 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:14.679822 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:14.679986 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:14.679890 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret podName:af6ceb0a-0f1f-42f9-8328-dc6263f46fe2 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:18.679860481 +0000 UTC m=+10.061389507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret") pod "global-pull-secret-syncer-nf6g7" (UID: "af6ceb0a-0f1f-42f9-8328-dc6263f46fe2") : object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:14.780228 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:14.780186 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fgfl\" (UniqueName: \"kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl\") pod \"network-check-target-525rm\" (UID: \"e9f0b4f6-e276-412b-9055-0389f82519db\") " pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:14.780453 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:14.780430 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 19:04:14.780544 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:14.780456 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 19:04:14.780544 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:14.780469 2563 projected.go:196] Error preparing data for projected volume kube-api-access-7fgfl for pod openshift-network-diagnostics/network-check-target-525rm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:14.780544 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:14.780533 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl podName:e9f0b4f6-e276-412b-9055-0389f82519db nodeName:}" failed. No retries permitted until 2026-09-13 19:04:18.780512663 +0000 UTC m=+10.162041695 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7fgfl" (UniqueName: "kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl") pod "network-check-target-525rm" (UID: "e9f0b4f6-e276-412b-9055-0389f82519db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:15.124931 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:15.124899 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:15.125126 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:15.125066 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:15.125398 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:15.125373 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:15.125796 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:15.125488 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:16.124475 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:16.124436 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:16.124666 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:16.124583 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:17.126965 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:17.126385 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:17.126965 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:17.126542 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:17.127519 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:17.127122 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:17.127519 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:17.127248 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:18.125081 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:18.124956 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:18.125277 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:18.125104 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:18.714062 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:18.713996 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:18.714062 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:18.714062 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:18.714799 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:18.714181 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:18.714799 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:18.714242 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:18.714799 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:18.714286 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret podName:af6ceb0a-0f1f-42f9-8328-dc6263f46fe2 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:26.714248324 +0000 UTC m=+18.095777351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret") pod "global-pull-secret-syncer-nf6g7" (UID: "af6ceb0a-0f1f-42f9-8328-dc6263f46fe2") : object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:18.714799 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:18.714322 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs podName:526ccac7-e62a-45c1-bc16-1c2f1bc7aca5 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:26.714303605 +0000 UTC m=+18.095832633 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs") pod "network-metrics-daemon-2wc2z" (UID: "526ccac7-e62a-45c1-bc16-1c2f1bc7aca5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:18.816182 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:18.815433 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fgfl\" (UniqueName: \"kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl\") pod \"network-check-target-525rm\" (UID: \"e9f0b4f6-e276-412b-9055-0389f82519db\") " pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:18.816182 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:18.815682 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 19:04:18.816182 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:18.815710 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 19:04:18.816182 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:18.815723 2563 projected.go:196] Error preparing data for projected volume kube-api-access-7fgfl for pod openshift-network-diagnostics/network-check-target-525rm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:18.816182 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:18.815793 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl podName:e9f0b4f6-e276-412b-9055-0389f82519db nodeName:}" failed. No retries permitted until 2026-09-13 19:04:26.815772926 +0000 UTC m=+18.197301955 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7fgfl" (UniqueName: "kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl") pod "network-check-target-525rm" (UID: "e9f0b4f6-e276-412b-9055-0389f82519db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:19.126371 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:19.126218 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:19.126561 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:19.126285 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:19.126645 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:19.126613 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:19.126752 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:19.126668 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:20.124233 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:20.124191 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:20.124743 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:20.124360 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:21.124656 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:21.124618 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:21.125019 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:21.124634 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:21.125019 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:21.124761 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:21.125019 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:21.124837 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:22.124768 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:22.124723 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:22.125163 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:22.124832 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:23.124992 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:23.124951 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:23.125644 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:23.125106 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:23.125644 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:23.124955 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:23.125644 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:23.125377 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:24.124966 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:24.124925 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:24.125145 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:24.125043 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:25.124857 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:25.124821 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:25.125066 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:25.124821 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:25.125066 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:25.124971 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:25.125066 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:25.125037 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:26.124114 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:26.124075 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:26.124371 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:26.124207 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:26.777971 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:26.777930 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:26.777971 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:26.777972 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:26.778410 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:26.778068 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:26.778410 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:26.778071 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:26.778410 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:26.778123 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs podName:526ccac7-e62a-45c1-bc16-1c2f1bc7aca5 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:42.778109641 +0000 UTC m=+34.159638648 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs") pod "network-metrics-daemon-2wc2z" (UID: "526ccac7-e62a-45c1-bc16-1c2f1bc7aca5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:26.778410 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:26.778135 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret podName:af6ceb0a-0f1f-42f9-8328-dc6263f46fe2 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:42.778129308 +0000 UTC m=+34.159658315 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret") pod "global-pull-secret-syncer-nf6g7" (UID: "af6ceb0a-0f1f-42f9-8328-dc6263f46fe2") : object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:26.878688 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:26.878651 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fgfl\" (UniqueName: \"kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl\") pod \"network-check-target-525rm\" (UID: \"e9f0b4f6-e276-412b-9055-0389f82519db\") " pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:26.878869 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:26.878830 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 19:04:26.878869 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:26.878851 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 19:04:26.878869 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:26.878861 2563 projected.go:196] Error preparing data for projected volume kube-api-access-7fgfl for pod openshift-network-diagnostics/network-check-target-525rm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:26.878957 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:26.878914 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl podName:e9f0b4f6-e276-412b-9055-0389f82519db nodeName:}" failed. No retries permitted until 2026-09-13 19:04:42.878901719 +0000 UTC m=+34.260430726 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7fgfl" (UniqueName: "kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl") pod "network-check-target-525rm" (UID: "e9f0b4f6-e276-412b-9055-0389f82519db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:27.124216 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:27.124124 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:27.124378 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:27.124320 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:27.124378 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:27.124353 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:27.124502 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:27.124481 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:28.124753 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:28.124712 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:28.125213 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:28.124825 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:29.125411 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:29.125373 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:29.125843 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:29.125511 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:29.125843 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:29.125567 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:29.125843 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:29.125657 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:30.124613 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:30.124576 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:30.124820 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:30.124708 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:31.124636 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.124167 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:31.125176 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.124167 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:31.125176 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:31.124771 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:31.125176 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:31.124860 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:31.326523 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.326491 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" event={"ID":"45a03def-1925-4d5b-b8fe-b1449cce31fe","Type":"ContainerStarted","Data":"3c96cf24224a51aac9b5611bd4ff3b55c656f16a966e4d8c904d1631dff23f9a"} Sep 13 19:04:31.328143 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.328109 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fflnp" event={"ID":"8738a08c-3f02-4fa0-b77c-51357c3eb253","Type":"ContainerStarted","Data":"d3d714839b78e3c6f15430f5519f4cce7f35b9d92adf0b631330d8c306b20792"} Sep 13 19:04:31.329797 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.329773 2563 generic.go:356] "Generic (PLEG): container finished" podID="60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c" containerID="839cc15d2ddb74818f00fa880e317c624aa9fc414ede42e753e59cb52c72c81e" exitCode=0 Sep 13 19:04:31.329928 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.329840 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fztdh" event={"ID":"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c","Type":"ContainerDied","Data":"839cc15d2ddb74818f00fa880e317c624aa9fc414ede42e753e59cb52c72c81e"} Sep 13 19:04:31.331576 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.331499 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lk24l" event={"ID":"56671f6f-f921-4342-a427-bb5d123e57a9","Type":"ContainerStarted","Data":"0901d776e91019e5fb9731e4707f1c0590a5ebaf8ec0353c09f5c8eb3438db9b"} Sep 13 19:04:31.338563 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.338519 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" event={"ID":"6b418bcb-8317-400a-adde-d604e722d1b8","Type":"ContainerStarted","Data":"c6c234ff1a5f795879053cbf3c12fef42a94b87030da2602abfb3c2c7c40e069"} Sep 13 19:04:31.338721 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.338634 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" event={"ID":"6b418bcb-8317-400a-adde-d604e722d1b8","Type":"ContainerStarted","Data":"79ea9e2ab2395262464369c92e109eb71c29c9f099ef2ce3b815ea045d1c39f7"} Sep 13 19:04:31.338721 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.338681 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" event={"ID":"6b418bcb-8317-400a-adde-d604e722d1b8","Type":"ContainerStarted","Data":"6993558b4dfd76478da495df5a72c6534a63c89763585747c1d2bf0a56b650c1"} Sep 13 19:04:31.338721 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.338712 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" event={"ID":"6b418bcb-8317-400a-adde-d604e722d1b8","Type":"ContainerStarted","Data":"baac33f1ceb8e19e7ec60a115a6614e5a5209ee73208588d291994573b42cc93"} Sep 13 19:04:31.338868 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.338746 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" event={"ID":"6b418bcb-8317-400a-adde-d604e722d1b8","Type":"ContainerStarted","Data":"07974243a28e4e1f5aad509c7824c836da3332ba76884cf67b5d2092b13c98bb"} Sep 13 19:04:31.338868 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.338786 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" event={"ID":"6b418bcb-8317-400a-adde-d604e722d1b8","Type":"ContainerStarted","Data":"c3a5fdd6fed40f1a6bb8f8e27804398bf9b2cd8648efb868548953d2a8639b5c"} Sep 13 19:04:31.340026 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.339997 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fbmxs" event={"ID":"effec463-bf64-4a1c-b7ad-9bee7e4901df","Type":"ContainerStarted","Data":"2f94dc9d37aae85eaa6d074b60fd45b062c16d9e968a1534dfc30aae47be1e3a"} Sep 13 19:04:31.341467 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.341414 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fflnp" podStartSLOduration=11.078672476 podStartE2EDuration="22.341395896s" totalImagesPullingTime="11.26272342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:04:31.341123195 +0000 UTC m=+22.722652223" watchObservedRunningTime="2026-09-13 19:04:31.341395896 +0000 UTC m=+22.722924926" Sep 13 19:04:31.341634 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.341604 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" podStartSLOduration=21.341594106 podStartE2EDuration="21.341594106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:04:14.303111116 +0000 UTC m=+5.684640149" watchObservedRunningTime="2026-09-13 19:04:31.341594106 +0000 UTC m=+22.723123135" Sep 13 19:04:31.342138 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.342112 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ttddh" event={"ID":"31fa23e6-3a87-4f8f-9f54-48853123eb54","Type":"ContainerStarted","Data":"a29afb36f148b8534db87dd602b6c4e5f52bb17dee6ee5c4925ed85745aa361f"} Sep 13 19:04:31.343678 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.343655 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" event={"ID":"80f433a2-abd8-460f-848a-91dda050f334","Type":"ContainerStarted","Data":"45ece73790e02d7f61257fa78d9d7198bc23dd961bebe9350e3d0d0b93380f30"} Sep 13 19:04:31.370959 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.370896 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lk24l" podStartSLOduration=3.80159106 podStartE2EDuration="22.370873983s" totalImagesPullingTime="18.569282923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:04:31.370725076 +0000 UTC m=+22.752254105" watchObservedRunningTime="2026-09-13 19:04:31.370873983 +0000 UTC m=+22.752403013" Sep 13 19:04:31.418019 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.417961 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t9kpc" podStartSLOduration=3.905170623 podStartE2EDuration="22.417946343s" totalImagesPullingTime="18.51277572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:04:31.417394558 +0000 UTC m=+22.798923598" watchObservedRunningTime="2026-09-13 19:04:31.417946343 +0000 UTC m=+22.799475414" Sep 13 19:04:31.418230 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.418080 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fbmxs" podStartSLOduration=3.909902695 podStartE2EDuration="22.418072898s" totalImagesPullingTime="18.508170203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:04:31.383220751 +0000 UTC m=+22.764749783" watchObservedRunningTime="2026-09-13 19:04:31.418072898 +0000 UTC m=+22.799601928" Sep 13 19:04:31.438671 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:31.438622 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ttddh" podStartSLOduration=3.940647527 podStartE2EDuration="22.438604356s" totalImagesPullingTime="18.497956829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:04:31.43802653 +0000 UTC m=+22.819555558" watchObservedRunningTime="2026-09-13 19:04:31.438604356 +0000 UTC m=+22.820133384" Sep 13 19:04:32.124412 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:32.124374 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:32.124589 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:32.124509 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:32.348893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:32.348463 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m2gng" event={"ID":"044cf94a-295f-4bef-905e-cac6f45bb36b","Type":"ContainerStarted","Data":"3e341784bf3434933a9152cd27d036c85fe489e19cf95068f8ba1f40bec906d8"} Sep 13 19:04:32.361809 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:32.361748 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m2gng" podStartSLOduration=4.8579737 podStartE2EDuration="23.361726062s" totalImagesPullingTime="18.503752362s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:04:32.361610505 +0000 UTC m=+23.743139538" watchObservedRunningTime="2026-09-13 19:04:32.361726062 +0000 UTC m=+23.743255092" Sep 13 19:04:32.444099 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:32.443885 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 13 19:04:33.108539 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:33.108403 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-13T19:04:32.444094675Z","UUID":"52991a0b-0cf9-463e-8ea9-d562f165d87f","Handler":null,"Name":"","Endpoint":""} Sep 13 19:04:33.110463 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:33.110436 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 13 19:04:33.110598 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:33.110474 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 13 19:04:33.124416 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:33.124346 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:33.124606 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:33.124478 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:33.124606 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:33.124517 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:33.124606 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:33.124599 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:33.353334 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:33.353294 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" event={"ID":"45a03def-1925-4d5b-b8fe-b1449cce31fe","Type":"ContainerStarted","Data":"a2197291d883ceb6f3a22eeddc84fef4c5436279a943c8e680cca50c6ac3a938"} Sep 13 19:04:34.124280 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:34.124224 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:34.124476 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:34.124382 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:34.225066 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:34.225023 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fflnp" Sep 13 19:04:34.225786 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:34.225766 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fflnp" Sep 13 19:04:34.358368 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:34.358319 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" event={"ID":"45a03def-1925-4d5b-b8fe-b1449cce31fe","Type":"ContainerStarted","Data":"dc591cbe27b1142cee8dde55dd7cae44fba7a952c688a393c01d1997e05b8a1d"} Sep 13 19:04:34.362457 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:34.362427 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" event={"ID":"6b418bcb-8317-400a-adde-d604e722d1b8","Type":"ContainerStarted","Data":"7006bc752adebac7a9a22dcd6e83320dfbf49997a11aaebd8d2638a7d4771fd0"} Sep 13 19:04:34.362689 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:34.362668 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fflnp" Sep 13 19:04:34.363172 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:34.363156 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fflnp" Sep 13 19:04:34.373703 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:34.373651 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2gqws" podStartSLOduration=4.4036378339999995 podStartE2EDuration="25.373634368s" totalImagesPullingTime="20.969996534s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:04:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:04:34.373388251 +0000 UTC m=+25.754917274" watchObservedRunningTime="2026-09-13 19:04:34.373634368 +0000 UTC m=+25.755163397" Sep 13 19:04:35.124537 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:35.124502 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:35.124832 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:35.124754 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:35.124832 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:35.124818 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:35.124962 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:35.124939 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:36.124705 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:36.124533 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:36.125058 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:36.124782 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:36.369146 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:36.369108 2563 generic.go:356] "Generic (PLEG): container finished" podID="60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c" containerID="8dee2c706715a19a8459d8fd36a1c316a28745ae3ae89e088b0e1d4adfe2d74e" exitCode=0 Sep 13 19:04:36.369349 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:36.369195 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fztdh" event={"ID":"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c","Type":"ContainerDied","Data":"8dee2c706715a19a8459d8fd36a1c316a28745ae3ae89e088b0e1d4adfe2d74e"} Sep 13 19:04:36.376207 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:36.376176 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" event={"ID":"6b418bcb-8317-400a-adde-d604e722d1b8","Type":"ContainerStarted","Data":"2e2594c599ea0d20c35cf9fc2551d79d0e775965fd410d3d5745272b010d11fd"} Sep 13 19:04:36.408315 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:36.408220 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" podStartSLOduration=8.850756417 podStartE2EDuration="27.408203526s" totalImagesPullingTime="18.557447109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:04:36.407726537 +0000 UTC m=+27.789255578" watchObservedRunningTime="2026-09-13 19:04:36.408203526 +0000 UTC m=+27.789732554" Sep 13 19:04:37.124314 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.124260 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:37.124584 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:37.124424 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:37.124584 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.124462 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:37.124696 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:37.124590 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:37.381719 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.381627 2563 generic.go:356] "Generic (PLEG): container finished" podID="60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c" containerID="2163db9a17c219f87ab5d26140ee6d3f4718f69c7fd676f01444927101183a5f" exitCode=0 Sep 13 19:04:37.381719 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.381705 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fztdh" event={"ID":"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c","Type":"ContainerDied","Data":"2163db9a17c219f87ab5d26140ee6d3f4718f69c7fd676f01444927101183a5f"} Sep 13 19:04:37.382523 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.381841 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 13 19:04:37.382523 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.382206 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:37.382523 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.382297 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:37.401833 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.401804 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:37.404912 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.404881 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:37.650524 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.650439 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nf6g7"] Sep 13 19:04:37.650690 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.650575 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:37.650690 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:37.650661 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:37.657516 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.657471 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2wc2z"] Sep 13 19:04:37.657705 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.657608 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:37.657771 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:37.657745 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:37.658353 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.658328 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-525rm"] Sep 13 19:04:37.658470 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:37.658455 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:37.658578 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:37.658558 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:38.387219 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:38.387180 2563 generic.go:356] "Generic (PLEG): container finished" podID="60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c" containerID="2f81b2606731cf6da928608de4fb76363cd55338d7f2a92896208c528a8d1a01" exitCode=0 Sep 13 19:04:38.387703 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:38.387269 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fztdh" event={"ID":"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c","Type":"ContainerDied","Data":"2f81b2606731cf6da928608de4fb76363cd55338d7f2a92896208c528a8d1a01"} Sep 13 19:04:38.387703 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:38.387372 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 13 19:04:39.125053 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:39.125021 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:39.125302 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:39.125115 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:39.125302 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:39.125169 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:39.125302 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:39.125283 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:39.389898 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:39.389806 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 13 19:04:40.124575 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:40.124541 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:40.124742 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:40.124682 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:40.475722 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:40.475678 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:04:40.476214 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:40.475952 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 13 19:04:40.488472 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:40.488431 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" podUID="6b418bcb-8317-400a-adde-d604e722d1b8" containerName="ovnkube-controller" probeResult="failure" output="" Sep 13 19:04:40.499315 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:40.499264 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" podUID="6b418bcb-8317-400a-adde-d604e722d1b8" containerName="ovnkube-controller" probeResult="failure" output="" Sep 13 19:04:41.125196 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:41.125154 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:41.125408 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:41.125183 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:41.125408 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:41.125333 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:41.125516 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:41.125402 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:42.124217 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:42.124136 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:42.124630 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:42.124304 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:04:42.795488 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:42.795440 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:42.795834 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:42.795507 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:42.795834 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:42.795601 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:42.795834 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:42.795683 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret podName:af6ceb0a-0f1f-42f9-8328-dc6263f46fe2 nodeName:}" failed. No retries permitted until 2026-09-13 19:05:14.79566167 +0000 UTC m=+66.177190686 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret") pod "global-pull-secret-syncer-nf6g7" (UID: "af6ceb0a-0f1f-42f9-8328-dc6263f46fe2") : object "kube-system"/"original-pull-secret" not registered Sep 13 19:04:42.795834 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:42.795685 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:42.795834 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:42.795761 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs podName:526ccac7-e62a-45c1-bc16-1c2f1bc7aca5 nodeName:}" failed. No retries permitted until 2026-09-13 19:05:14.795746911 +0000 UTC m=+66.177275938 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs") pod "network-metrics-daemon-2wc2z" (UID: "526ccac7-e62a-45c1-bc16-1c2f1bc7aca5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 19:04:42.896476 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:42.896432 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fgfl\" (UniqueName: \"kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl\") pod \"network-check-target-525rm\" (UID: \"e9f0b4f6-e276-412b-9055-0389f82519db\") " pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:42.896649 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:42.896560 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 19:04:42.896649 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:42.896574 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 19:04:42.896649 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:42.896583 2563 projected.go:196] Error preparing data for projected volume kube-api-access-7fgfl for pod openshift-network-diagnostics/network-check-target-525rm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:42.896649 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:42.896631 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl podName:e9f0b4f6-e276-412b-9055-0389f82519db nodeName:}" failed. No retries permitted until 2026-09-13 19:05:14.89661826 +0000 UTC m=+66.278147280 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7fgfl" (UniqueName: "kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl") pod "network-check-target-525rm" (UID: "e9f0b4f6-e276-412b-9055-0389f82519db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 19:04:43.124687 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.124600 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:43.125055 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.124600 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:43.125055 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:43.124737 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-525rm" podUID="e9f0b4f6-e276-412b-9055-0389f82519db" Sep 13 19:04:43.125055 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:43.124778 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nf6g7" podUID="af6ceb0a-0f1f-42f9-8328-dc6263f46fe2" Sep 13 19:04:43.404216 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.404030 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fztdh" event={"ID":"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c","Type":"ContainerStarted","Data":"74d5175cd571ccd55d96b4fef0674cf421ce9c0f44cc469d1caa4935cea160d3"} Sep 13 19:04:43.445382 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.445348 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeReady" Sep 13 19:04:43.445537 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.445479 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 13 19:04:43.492808 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.492775 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bvg5t"] Sep 13 19:04:43.516861 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.516820 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4zfmc"] Sep 13 19:04:43.517021 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.516958 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:43.519419 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.519394 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kkpnt\"" Sep 13 19:04:43.519725 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.519701 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 13 19:04:43.519992 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.519976 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 13 19:04:43.539823 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.539782 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bvg5t"] Sep 13 19:04:43.539823 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.539823 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4zfmc"] Sep 13 19:04:43.540055 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.539917 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4zfmc" Sep 13 19:04:43.542868 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.542844 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-b8nhs\"" Sep 13 19:04:43.542868 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.542856 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 13 19:04:43.543050 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.542854 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 13 19:04:43.543188 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.543175 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 13 19:04:43.599591 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.599550 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b17a3853-b777-4980-9a42-0e87a2c963f8-config-volume\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:43.599789 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.599612 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26862\" (UniqueName: \"kubernetes.io/projected/b17a3853-b777-4980-9a42-0e87a2c963f8-kube-api-access-26862\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:43.599789 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.599663 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b17a3853-b777-4980-9a42-0e87a2c963f8-tmp-dir\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:43.599789 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.599716 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:43.700974 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.700935 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26862\" (UniqueName: \"kubernetes.io/projected/b17a3853-b777-4980-9a42-0e87a2c963f8-kube-api-access-26862\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:43.701089 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.700979 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327-cert\") pod \"ingress-canary-4zfmc\" (UID: \"bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327\") " pod="openshift-ingress-canary/ingress-canary-4zfmc" Sep 13 19:04:43.701089 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.701001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b17a3853-b777-4980-9a42-0e87a2c963f8-tmp-dir\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:43.701089 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.701044 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:43.701089 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.701062 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvz9x\" (UniqueName: \"kubernetes.io/projected/bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327-kube-api-access-zvz9x\") pod \"ingress-canary-4zfmc\" (UID: \"bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327\") " pod="openshift-ingress-canary/ingress-canary-4zfmc" Sep 13 19:04:43.701328 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.701097 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b17a3853-b777-4980-9a42-0e87a2c963f8-config-volume\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:43.701328 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:43.701192 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 19:04:43.701328 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:43.701291 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls podName:b17a3853-b777-4980-9a42-0e87a2c963f8 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:44.201268322 +0000 UTC m=+35.582797332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls") pod "dns-default-bvg5t" (UID: "b17a3853-b777-4980-9a42-0e87a2c963f8") : secret "dns-default-metrics-tls" not found Sep 13 19:04:43.701466 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.701445 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b17a3853-b777-4980-9a42-0e87a2c963f8-tmp-dir\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:43.701696 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.701681 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b17a3853-b777-4980-9a42-0e87a2c963f8-config-volume\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:43.711208 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.711168 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26862\" (UniqueName: \"kubernetes.io/projected/b17a3853-b777-4980-9a42-0e87a2c963f8-kube-api-access-26862\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:43.801810 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.801768 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvz9x\" (UniqueName: \"kubernetes.io/projected/bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327-kube-api-access-zvz9x\") pod \"ingress-canary-4zfmc\" (UID: \"bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327\") " pod="openshift-ingress-canary/ingress-canary-4zfmc" Sep 13 19:04:43.801982 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.801843 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327-cert\") pod \"ingress-canary-4zfmc\" (UID: \"bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327\") " pod="openshift-ingress-canary/ingress-canary-4zfmc" Sep 13 19:04:43.804555 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.804520 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327-cert\") pod \"ingress-canary-4zfmc\" (UID: \"bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327\") " pod="openshift-ingress-canary/ingress-canary-4zfmc" Sep 13 19:04:43.810257 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.810214 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvz9x\" (UniqueName: \"kubernetes.io/projected/bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327-kube-api-access-zvz9x\") pod \"ingress-canary-4zfmc\" (UID: \"bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327\") " pod="openshift-ingress-canary/ingress-canary-4zfmc" Sep 13 19:04:43.859213 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:43.859170 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4zfmc" Sep 13 19:04:44.124434 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:44.124391 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:04:44.127334 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:44.127304 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 13 19:04:44.127990 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:44.127350 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-454nd\"" Sep 13 19:04:44.192767 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:44.192722 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4zfmc"] Sep 13 19:04:44.197636 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:04:44.197604 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc0cbcda_2f94_4fd9_bfb9_f2d4454a9327.slice/crio-c87a88972b53bced0c3ee7e560fd6c45a0fc705ad45d584fd09dea91ec142436 WatchSource:0}: Error finding container c87a88972b53bced0c3ee7e560fd6c45a0fc705ad45d584fd09dea91ec142436: Status 404 returned error can't find the container with id c87a88972b53bced0c3ee7e560fd6c45a0fc705ad45d584fd09dea91ec142436 Sep 13 19:04:44.204705 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:44.204679 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:44.204817 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:44.204787 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 19:04:44.204894 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:44.204883 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls podName:b17a3853-b777-4980-9a42-0e87a2c963f8 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:45.204865376 +0000 UTC m=+36.586394386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls") pod "dns-default-bvg5t" (UID: "b17a3853-b777-4980-9a42-0e87a2c963f8") : secret "dns-default-metrics-tls" not found Sep 13 19:04:44.409636 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:44.409541 2563 generic.go:356] "Generic (PLEG): container finished" podID="60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c" containerID="74d5175cd571ccd55d96b4fef0674cf421ce9c0f44cc469d1caa4935cea160d3" exitCode=0 Sep 13 19:04:44.409794 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:44.409630 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fztdh" event={"ID":"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c","Type":"ContainerDied","Data":"74d5175cd571ccd55d96b4fef0674cf421ce9c0f44cc469d1caa4935cea160d3"} Sep 13 19:04:44.410806 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:44.410784 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4zfmc" event={"ID":"bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327","Type":"ContainerStarted","Data":"c87a88972b53bced0c3ee7e560fd6c45a0fc705ad45d584fd09dea91ec142436"} Sep 13 19:04:45.125199 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:45.124961 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:04:45.125385 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:45.125052 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:04:45.127575 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:45.127552 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 13 19:04:45.127974 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:45.127909 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 13 19:04:45.127974 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:45.127947 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cs6ck\"" Sep 13 19:04:45.128092 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:45.127978 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 13 19:04:45.212559 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:45.212523 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:45.212729 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:45.212662 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 19:04:45.212781 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:45.212739 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls podName:b17a3853-b777-4980-9a42-0e87a2c963f8 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:47.21271913 +0000 UTC m=+38.594248152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls") pod "dns-default-bvg5t" (UID: "b17a3853-b777-4980-9a42-0e87a2c963f8") : secret "dns-default-metrics-tls" not found Sep 13 19:04:45.417896 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:45.417807 2563 generic.go:356] "Generic (PLEG): container finished" podID="60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c" containerID="d5115a1bf76678869f55bd0d4c88e6e7466af7cc9b74ec7df6dcabc7d87ffe15" exitCode=0 Sep 13 19:04:45.417896 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:45.417850 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fztdh" event={"ID":"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c","Type":"ContainerDied","Data":"d5115a1bf76678869f55bd0d4c88e6e7466af7cc9b74ec7df6dcabc7d87ffe15"} Sep 13 19:04:46.424932 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:46.424891 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fztdh" event={"ID":"60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c","Type":"ContainerStarted","Data":"a7f5a5a368fcb7f32269b13b92f257757e10b90817cf80956736219b69697ef2"} Sep 13 19:04:46.446376 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:46.446309 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fztdh" podStartSLOduration=7.925719483 podStartE2EDuration="37.446289555s" totalImagesPullingTime="29.520570072s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:04:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:04:46.445050497 +0000 UTC m=+37.826579526" watchObservedRunningTime="2026-09-13 19:04:46.446289555 +0000 UTC m=+37.827818584" Sep 13 19:04:47.226448 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:47.226405 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:47.226648 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:47.226576 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 19:04:47.226716 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:47.226658 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls podName:b17a3853-b777-4980-9a42-0e87a2c963f8 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:51.226636544 +0000 UTC m=+42.608165550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls") pod "dns-default-bvg5t" (UID: "b17a3853-b777-4980-9a42-0e87a2c963f8") : secret "dns-default-metrics-tls" not found Sep 13 19:04:47.428976 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:47.428938 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4zfmc" event={"ID":"bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327","Type":"ContainerStarted","Data":"aeab3312cd611c0b3d2c8a3356b030b9c97863e2634d5fb6def31d8c00d85e35"} Sep 13 19:04:47.445796 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:47.445735 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4zfmc" podStartSLOduration=1.687097911 podStartE2EDuration="4.445715426s" totalImagesPullingTime="2.758617515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:04:47.445263613 +0000 UTC m=+38.826792633" watchObservedRunningTime="2026-09-13 19:04:47.445715426 +0000 UTC m=+38.827244456" Sep 13 19:04:51.251195 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:51.251148 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:51.251605 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:51.251339 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 19:04:51.251605 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:51.251410 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls podName:b17a3853-b777-4980-9a42-0e87a2c963f8 nodeName:}" failed. No retries permitted until 2026-09-13 19:04:59.251393458 +0000 UTC m=+50.632922470 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls") pod "dns-default-bvg5t" (UID: "b17a3853-b777-4980-9a42-0e87a2c963f8") : secret "dns-default-metrics-tls" not found Sep 13 19:04:59.311598 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:04:59.311551 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:04:59.312043 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:59.311709 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 19:04:59.312043 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:04:59.311776 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls podName:b17a3853-b777-4980-9a42-0e87a2c963f8 nodeName:}" failed. No retries permitted until 2026-09-13 19:05:15.311759465 +0000 UTC m=+66.693288475 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls") pod "dns-default-bvg5t" (UID: "b17a3853-b777-4980-9a42-0e87a2c963f8") : secret "dns-default-metrics-tls" not found Sep 13 19:05:10.498576 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:10.498547 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tpxtd" Sep 13 19:05:14.822510 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:14.822467 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:05:14.822903 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:14.822520 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:05:14.828809 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:14.828783 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 13 19:05:14.828871 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:14.828841 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 13 19:05:14.832832 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:05:14.832805 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 13 19:05:14.832909 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:05:14.832898 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs podName:526ccac7-e62a-45c1-bc16-1c2f1bc7aca5 nodeName:}" failed. No retries permitted until 2026-09-13 19:06:18.832878977 +0000 UTC m=+130.214408004 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs") pod "network-metrics-daemon-2wc2z" (UID: "526ccac7-e62a-45c1-bc16-1c2f1bc7aca5") : secret "metrics-daemon-secret" not found Sep 13 19:05:14.836050 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:14.836017 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af6ceb0a-0f1f-42f9-8328-dc6263f46fe2-original-pull-secret\") pod \"global-pull-secret-syncer-nf6g7\" (UID: \"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2\") " pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:05:14.837482 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:14.837461 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nf6g7" Sep 13 19:05:14.923736 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:14.923690 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fgfl\" (UniqueName: \"kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl\") pod \"network-check-target-525rm\" (UID: \"e9f0b4f6-e276-412b-9055-0389f82519db\") " pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:05:14.926140 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:14.926117 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 13 19:05:14.936161 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:14.936136 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 13 19:05:14.947612 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:14.947586 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fgfl\" (UniqueName: \"kubernetes.io/projected/e9f0b4f6-e276-412b-9055-0389f82519db-kube-api-access-7fgfl\") pod \"network-check-target-525rm\" (UID: \"e9f0b4f6-e276-412b-9055-0389f82519db\") " pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:05:15.105230 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:15.105110 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nf6g7"] Sep 13 19:05:15.109551 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:05:15.109521 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6ceb0a_0f1f_42f9_8328_dc6263f46fe2.slice/crio-c6487d5be12f9d279ef54aa81f72d9975265e389a708a55f4674be8c52d5b999 WatchSource:0}: Error finding container c6487d5be12f9d279ef54aa81f72d9975265e389a708a55f4674be8c52d5b999: Status 404 returned error can't find the container with id c6487d5be12f9d279ef54aa81f72d9975265e389a708a55f4674be8c52d5b999 Sep 13 19:05:15.146847 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:15.146813 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cs6ck\"" Sep 13 19:05:15.154580 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:15.154555 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:05:15.326053 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:15.326002 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:05:15.326232 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:05:15.326153 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 19:05:15.326232 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:05:15.326222 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls podName:b17a3853-b777-4980-9a42-0e87a2c963f8 nodeName:}" failed. No retries permitted until 2026-09-13 19:05:47.326204289 +0000 UTC m=+98.707733296 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls") pod "dns-default-bvg5t" (UID: "b17a3853-b777-4980-9a42-0e87a2c963f8") : secret "dns-default-metrics-tls" not found Sep 13 19:05:15.421517 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:15.421439 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-525rm"] Sep 13 19:05:15.424791 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:05:15.424764 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f0b4f6_e276_412b_9055_0389f82519db.slice/crio-8c3a83d5ffe0cbc778c73a621af17a238043477c4167516be0d6746cd2ac7f28 WatchSource:0}: Error finding container 8c3a83d5ffe0cbc778c73a621af17a238043477c4167516be0d6746cd2ac7f28: Status 404 returned error can't find the container with id 8c3a83d5ffe0cbc778c73a621af17a238043477c4167516be0d6746cd2ac7f28 Sep 13 19:05:15.498130 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:15.498091 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-525rm" event={"ID":"e9f0b4f6-e276-412b-9055-0389f82519db","Type":"ContainerStarted","Data":"8c3a83d5ffe0cbc778c73a621af17a238043477c4167516be0d6746cd2ac7f28"} Sep 13 19:05:15.499163 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:15.499132 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nf6g7" event={"ID":"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2","Type":"ContainerStarted","Data":"c6487d5be12f9d279ef54aa81f72d9975265e389a708a55f4674be8c52d5b999"} Sep 13 19:05:19.128683 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.128644 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr"] Sep 13 19:05:19.130815 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.130787 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:19.134224 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.134169 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 13 19:05:19.134659 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.134628 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 13 19:05:19.135526 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.135505 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 13 19:05:19.140682 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.140659 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 13 19:05:19.151811 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.151778 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr"] Sep 13 19:05:19.158710 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.158676 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgsmr\" (UniqueName: \"kubernetes.io/projected/325b1453-ad30-4910-85fb-31e71f55c517-kube-api-access-xgsmr\") pod \"klusterlet-addon-workmgr-7fc9f97bb7-zc4sr\" (UID: \"325b1453-ad30-4910-85fb-31e71f55c517\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:19.158879 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.158720 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/325b1453-ad30-4910-85fb-31e71f55c517-klusterlet-config\") pod \"klusterlet-addon-workmgr-7fc9f97bb7-zc4sr\" (UID: \"325b1453-ad30-4910-85fb-31e71f55c517\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:19.158934 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.158876 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/325b1453-ad30-4910-85fb-31e71f55c517-tmp\") pod \"klusterlet-addon-workmgr-7fc9f97bb7-zc4sr\" (UID: \"325b1453-ad30-4910-85fb-31e71f55c517\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:19.259985 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.259942 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgsmr\" (UniqueName: \"kubernetes.io/projected/325b1453-ad30-4910-85fb-31e71f55c517-kube-api-access-xgsmr\") pod \"klusterlet-addon-workmgr-7fc9f97bb7-zc4sr\" (UID: \"325b1453-ad30-4910-85fb-31e71f55c517\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:19.260194 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.259997 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/325b1453-ad30-4910-85fb-31e71f55c517-klusterlet-config\") pod \"klusterlet-addon-workmgr-7fc9f97bb7-zc4sr\" (UID: \"325b1453-ad30-4910-85fb-31e71f55c517\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:19.260194 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.260091 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/325b1453-ad30-4910-85fb-31e71f55c517-tmp\") pod \"klusterlet-addon-workmgr-7fc9f97bb7-zc4sr\" (UID: \"325b1453-ad30-4910-85fb-31e71f55c517\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:19.260579 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.260552 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/325b1453-ad30-4910-85fb-31e71f55c517-tmp\") pod \"klusterlet-addon-workmgr-7fc9f97bb7-zc4sr\" (UID: \"325b1453-ad30-4910-85fb-31e71f55c517\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:19.263421 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.263390 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/325b1453-ad30-4910-85fb-31e71f55c517-klusterlet-config\") pod \"klusterlet-addon-workmgr-7fc9f97bb7-zc4sr\" (UID: \"325b1453-ad30-4910-85fb-31e71f55c517\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:19.268396 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.268367 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgsmr\" (UniqueName: \"kubernetes.io/projected/325b1453-ad30-4910-85fb-31e71f55c517-kube-api-access-xgsmr\") pod \"klusterlet-addon-workmgr-7fc9f97bb7-zc4sr\" (UID: \"325b1453-ad30-4910-85fb-31e71f55c517\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:19.444742 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:19.444637 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:20.208293 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:20.208186 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr"] Sep 13 19:05:20.211417 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:05:20.211387 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325b1453_ad30_4910_85fb_31e71f55c517.slice/crio-a417ef2c0640bc14b8cf8a36e492724baba846df5cf2d17a611506a1e1d5add1 WatchSource:0}: Error finding container a417ef2c0640bc14b8cf8a36e492724baba846df5cf2d17a611506a1e1d5add1: Status 404 returned error can't find the container with id a417ef2c0640bc14b8cf8a36e492724baba846df5cf2d17a611506a1e1d5add1 Sep 13 19:05:20.514767 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:20.514725 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" event={"ID":"325b1453-ad30-4910-85fb-31e71f55c517","Type":"ContainerStarted","Data":"a417ef2c0640bc14b8cf8a36e492724baba846df5cf2d17a611506a1e1d5add1"} Sep 13 19:05:20.516149 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:20.516125 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-525rm" event={"ID":"e9f0b4f6-e276-412b-9055-0389f82519db","Type":"ContainerStarted","Data":"573f6ee8c6ef8058c0cca7c91fc863fb045ba2c61130f3748a12d0003ff259e9"} Sep 13 19:05:20.516292 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:20.516267 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:05:20.517566 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:20.517542 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nf6g7" event={"ID":"af6ceb0a-0f1f-42f9-8328-dc6263f46fe2","Type":"ContainerStarted","Data":"9f52eb46390b4d425c8a6fcd799d37a6a2b0298d2f894e0593e8d6d7cdf5fe3a"} Sep 13 19:05:20.530902 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:20.530843 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-525rm" podStartSLOduration=66.007138395 podStartE2EDuration="1m10.530827756s" totalImagesPullingTime="4.523689361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:05:20.530443828 +0000 UTC m=+71.911972857" watchObservedRunningTime="2026-09-13 19:05:20.530827756 +0000 UTC m=+71.912356786" Sep 13 19:05:20.545712 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:20.545653 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nf6g7" podStartSLOduration=66.702658657 podStartE2EDuration="1m11.545636352s" totalImagesPullingTime="4.842977695s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:05:20.54501385 +0000 UTC m=+71.926542873" watchObservedRunningTime="2026-09-13 19:05:20.545636352 +0000 UTC m=+71.927165380" Sep 13 19:05:24.530276 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:24.530220 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" event={"ID":"325b1453-ad30-4910-85fb-31e71f55c517","Type":"ContainerStarted","Data":"d611f8422e393b21dd69efcfda98269e9c7f07133fee2a00829d990928ac410e"} Sep 13 19:05:24.530703 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:24.530439 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:24.532058 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:24.532033 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:05:24.545695 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:24.545642 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" podStartSLOduration=1.9460029570000001 podStartE2EDuration="5.54562638s" totalImagesPullingTime="3.599623423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:05:24.544708079 +0000 UTC m=+75.926237111" watchObservedRunningTime="2026-09-13 19:05:24.54562638 +0000 UTC m=+75.927155409" Sep 13 19:05:47.355310 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:47.355267 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:05:47.355763 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:05:47.355397 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 19:05:47.355763 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:05:47.355464 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls podName:b17a3853-b777-4980-9a42-0e87a2c963f8 nodeName:}" failed. No retries permitted until 2026-09-13 19:06:51.355443709 +0000 UTC m=+162.736972731 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls") pod "dns-default-bvg5t" (UID: "b17a3853-b777-4980-9a42-0e87a2c963f8") : secret "dns-default-metrics-tls" not found Sep 13 19:05:51.523051 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:05:51.523021 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-525rm" Sep 13 19:06:18.871431 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:06:18.871369 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:06:18.871925 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:06:18.871536 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 13 19:06:18.871925 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:06:18.871641 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs podName:526ccac7-e62a-45c1-bc16-1c2f1bc7aca5 nodeName:}" failed. No retries permitted until 2026-09-13 19:08:20.871624003 +0000 UTC m=+252.253153015 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs") pod "network-metrics-daemon-2wc2z" (UID: "526ccac7-e62a-45c1-bc16-1c2f1bc7aca5") : secret "metrics-daemon-secret" not found Sep 13 19:06:46.526464 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:06:46.526419 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bvg5t" podUID="b17a3853-b777-4980-9a42-0e87a2c963f8" Sep 13 19:06:46.735782 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:06:46.735752 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bvg5t" Sep 13 19:06:47.133746 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:06:47.133694 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-2wc2z" podUID="526ccac7-e62a-45c1-bc16-1c2f1bc7aca5" Sep 13 19:06:51.397418 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:06:51.397350 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:06:51.397841 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:06:51.397513 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 19:06:51.397841 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:06:51.397598 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls podName:b17a3853-b777-4980-9a42-0e87a2c963f8 nodeName:}" failed. No retries permitted until 2026-09-13 19:08:53.397580756 +0000 UTC m=+284.779109766 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls") pod "dns-default-bvg5t" (UID: "b17a3853-b777-4980-9a42-0e87a2c963f8") : secret "dns-default-metrics-tls" not found Sep 13 19:06:58.124361 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:06:58.124310 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:07:24.531296 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:07:24.531227 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" podUID="325b1453-ad30-4910-85fb-31e71f55c517" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused" Sep 13 19:07:24.831965 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:07:24.831877 2563 generic.go:356] "Generic (PLEG): container finished" podID="325b1453-ad30-4910-85fb-31e71f55c517" containerID="d611f8422e393b21dd69efcfda98269e9c7f07133fee2a00829d990928ac410e" exitCode=1 Sep 13 19:07:24.831965 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:07:24.831928 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" event={"ID":"325b1453-ad30-4910-85fb-31e71f55c517","Type":"ContainerDied","Data":"d611f8422e393b21dd69efcfda98269e9c7f07133fee2a00829d990928ac410e"} Sep 13 19:07:24.832300 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:07:24.832285 2563 scope.go:117] "RemoveContainer" containerID="d611f8422e393b21dd69efcfda98269e9c7f07133fee2a00829d990928ac410e" Sep 13 19:07:25.835884 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:07:25.835849 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" event={"ID":"325b1453-ad30-4910-85fb-31e71f55c517","Type":"ContainerStarted","Data":"8f88dc5487c00bbee9c192f43ff93e3748e6805e9c96f30019f0af903b025056"} Sep 13 19:07:25.836270 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:07:25.836129 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:07:25.836754 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:07:25.836735 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fc9f97bb7-zc4sr" Sep 13 19:08:02.664710 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.664674 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf"] Sep 13 19:08:02.666630 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.666610 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-875dw"] Sep 13 19:08:02.666748 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.666728 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" Sep 13 19:08:02.668456 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.668424 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.669553 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.669530 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-j7hrj\"" Sep 13 19:08:02.669773 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.669757 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 13 19:08:02.669935 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.669918 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 13 19:08:02.669976 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.669950 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 13 19:08:02.670353 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.670336 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 13 19:08:02.670997 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.670980 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 13 19:08:02.671067 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.670985 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 13 19:08:02.671729 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.671712 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 13 19:08:02.671808 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.671759 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 13 19:08:02.672466 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.672449 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ngnbj\"" Sep 13 19:08:02.676748 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.676728 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 13 19:08:02.689644 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.689610 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-875dw"] Sep 13 19:08:02.690361 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.690338 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf"] Sep 13 19:08:02.760539 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.760494 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-q48gb"] Sep 13 19:08:02.762692 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.762664 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd"] Sep 13 19:08:02.762857 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.762788 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-q48gb" Sep 13 19:08:02.764655 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.764630 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59698f7bd5-xn44h"] Sep 13 19:08:02.764803 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.764741 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:02.765336 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.765308 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jmxlt\"" Sep 13 19:08:02.766440 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.766422 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.767789 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.767765 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 13 19:08:02.767789 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.767777 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 13 19:08:02.767953 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.767774 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-g767s\"" Sep 13 19:08:02.767953 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.767820 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 13 19:08:02.767953 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.767770 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 13 19:08:02.768855 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.768839 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 13 19:08:02.769154 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.769140 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 13 19:08:02.769428 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.769413 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mg4q7\"" Sep 13 19:08:02.770094 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.770073 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 13 19:08:02.775555 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.775526 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-q48gb"] Sep 13 19:08:02.775963 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.775939 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 13 19:08:02.776533 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.776512 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd"] Sep 13 19:08:02.778990 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.778958 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59698f7bd5-xn44h"] Sep 13 19:08:02.794725 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.794686 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f8bt\" (UniqueName: \"kubernetes.io/projected/892493a1-d1c1-4f71-9621-77beb61a4a95-kube-api-access-2f8bt\") pod \"console-operator-696884fb9c-875dw\" (UID: \"892493a1-d1c1-4f71-9621-77beb61a4a95\") " pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.794916 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.794739 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvlws\" (UniqueName: \"kubernetes.io/projected/cc5e33f3-8e38-43dc-9f04-bef15c0d860c-kube-api-access-kvlws\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nsbnf\" (UID: \"cc5e33f3-8e38-43dc-9f04-bef15c0d860c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" Sep 13 19:08:02.794916 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.794761 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/892493a1-d1c1-4f71-9621-77beb61a4a95-trusted-ca\") pod \"console-operator-696884fb9c-875dw\" (UID: \"892493a1-d1c1-4f71-9621-77beb61a4a95\") " pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.794916 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.794819 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/892493a1-d1c1-4f71-9621-77beb61a4a95-serving-cert\") pod \"console-operator-696884fb9c-875dw\" (UID: \"892493a1-d1c1-4f71-9621-77beb61a4a95\") " pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.794916 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.794837 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cc5e33f3-8e38-43dc-9f04-bef15c0d860c-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nsbnf\" (UID: \"cc5e33f3-8e38-43dc-9f04-bef15c0d860c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" Sep 13 19:08:02.794916 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.794863 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/892493a1-d1c1-4f71-9621-77beb61a4a95-config\") pod \"console-operator-696884fb9c-875dw\" (UID: \"892493a1-d1c1-4f71-9621-77beb61a4a95\") " pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.794916 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.794885 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cc5e33f3-8e38-43dc-9f04-bef15c0d860c-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nsbnf\" (UID: \"cc5e33f3-8e38-43dc-9f04-bef15c0d860c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" Sep 13 19:08:02.895561 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895522 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cc5e33f3-8e38-43dc-9f04-bef15c0d860c-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nsbnf\" (UID: \"cc5e33f3-8e38-43dc-9f04-bef15c0d860c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" Sep 13 19:08:02.895561 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895564 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f8bt\" (UniqueName: \"kubernetes.io/projected/892493a1-d1c1-4f71-9621-77beb61a4a95-kube-api-access-2f8bt\") pod \"console-operator-696884fb9c-875dw\" (UID: \"892493a1-d1c1-4f71-9621-77beb61a4a95\") " pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.895831 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895588 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:02.895831 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895607 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgksk\" (UniqueName: \"kubernetes.io/projected/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-kube-api-access-kgksk\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:02.895831 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895636 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-certificates\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.895831 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895667 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsrcb\" (UniqueName: \"kubernetes.io/projected/b4d96614-aac8-4a7e-86b1-007714a7a27f-kube-api-access-hsrcb\") pod \"network-check-source-6f8b768455-q48gb\" (UID: \"b4d96614-aac8-4a7e-86b1-007714a7a27f\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-q48gb" Sep 13 19:08:02.895831 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895747 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-ca-trust-extracted\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.895831 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895778 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvlws\" (UniqueName: \"kubernetes.io/projected/cc5e33f3-8e38-43dc-9f04-bef15c0d860c-kube-api-access-kvlws\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nsbnf\" (UID: \"cc5e33f3-8e38-43dc-9f04-bef15c0d860c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" Sep 13 19:08:02.895831 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895805 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/892493a1-d1c1-4f71-9621-77beb61a4a95-trusted-ca\") pod \"console-operator-696884fb9c-875dw\" (UID: \"892493a1-d1c1-4f71-9621-77beb61a4a95\") " pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.895831 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895830 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-image-registry-private-configuration\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.896214 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895859 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szt7z\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-kube-api-access-szt7z\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.896214 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895899 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-trusted-ca\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.896214 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895928 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-bound-sa-token\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.896214 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895959 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/892493a1-d1c1-4f71-9621-77beb61a4a95-serving-cert\") pod \"console-operator-696884fb9c-875dw\" (UID: \"892493a1-d1c1-4f71-9621-77beb61a4a95\") " pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.896214 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.895985 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.896214 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.896018 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cc5e33f3-8e38-43dc-9f04-bef15c0d860c-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nsbnf\" (UID: \"cc5e33f3-8e38-43dc-9f04-bef15c0d860c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" Sep 13 19:08:02.896214 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.896061 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:02.896214 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.896120 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/892493a1-d1c1-4f71-9621-77beb61a4a95-config\") pod \"console-operator-696884fb9c-875dw\" (UID: \"892493a1-d1c1-4f71-9621-77beb61a4a95\") " pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.896214 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.896162 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-installation-pull-secrets\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.896581 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.896366 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cc5e33f3-8e38-43dc-9f04-bef15c0d860c-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nsbnf\" (UID: \"cc5e33f3-8e38-43dc-9f04-bef15c0d860c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" Sep 13 19:08:02.896754 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.896734 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/892493a1-d1c1-4f71-9621-77beb61a4a95-config\") pod \"console-operator-696884fb9c-875dw\" (UID: \"892493a1-d1c1-4f71-9621-77beb61a4a95\") " pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.896817 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.896798 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/892493a1-d1c1-4f71-9621-77beb61a4a95-trusted-ca\") pod \"console-operator-696884fb9c-875dw\" (UID: \"892493a1-d1c1-4f71-9621-77beb61a4a95\") " pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.898374 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.898351 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cc5e33f3-8e38-43dc-9f04-bef15c0d860c-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nsbnf\" (UID: \"cc5e33f3-8e38-43dc-9f04-bef15c0d860c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" Sep 13 19:08:02.898468 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.898427 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/892493a1-d1c1-4f71-9621-77beb61a4a95-serving-cert\") pod \"console-operator-696884fb9c-875dw\" (UID: \"892493a1-d1c1-4f71-9621-77beb61a4a95\") " pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.903379 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.903351 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f8bt\" (UniqueName: \"kubernetes.io/projected/892493a1-d1c1-4f71-9621-77beb61a4a95-kube-api-access-2f8bt\") pod \"console-operator-696884fb9c-875dw\" (UID: \"892493a1-d1c1-4f71-9621-77beb61a4a95\") " pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.903503 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.903483 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvlws\" (UniqueName: \"kubernetes.io/projected/cc5e33f3-8e38-43dc-9f04-bef15c0d860c-kube-api-access-kvlws\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nsbnf\" (UID: \"cc5e33f3-8e38-43dc-9f04-bef15c0d860c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" Sep 13 19:08:02.977604 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.977568 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" Sep 13 19:08:02.982973 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.982944 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:02.996803 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.996764 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-image-registry-private-configuration\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.996803 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.996803 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szt7z\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-kube-api-access-szt7z\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.997045 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.996828 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-trusted-ca\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.997045 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.996847 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-bound-sa-token\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.997045 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.996873 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.997045 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.996899 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:02.997045 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.996953 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-installation-pull-secrets\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.997045 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.997009 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:02.997045 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.997035 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgksk\" (UniqueName: \"kubernetes.io/projected/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-kube-api-access-kgksk\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:02.997417 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.997064 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-certificates\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.997417 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:02.997070 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 19:08:02.997417 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:02.997088 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59698f7bd5-xn44h: secret "image-registry-tls" not found Sep 13 19:08:02.997417 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.997091 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsrcb\" (UniqueName: \"kubernetes.io/projected/b4d96614-aac8-4a7e-86b1-007714a7a27f-kube-api-access-hsrcb\") pod \"network-check-source-6f8b768455-q48gb\" (UID: \"b4d96614-aac8-4a7e-86b1-007714a7a27f\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-q48gb" Sep 13 19:08:02.997417 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.997132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-ca-trust-extracted\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.997417 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:02.997156 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls podName:7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48 nodeName:}" failed. No retries permitted until 2026-09-13 19:08:03.497136165 +0000 UTC m=+234.878665191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls") pod "image-registry-59698f7bd5-xn44h" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48") : secret "image-registry-tls" not found Sep 13 19:08:02.997417 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:02.997340 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 13 19:08:02.997417 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:02.997400 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls podName:6e970a71-b45e-4fdc-a726-6a1e3b97e91c nodeName:}" failed. No retries permitted until 2026-09-13 19:08:03.497380146 +0000 UTC m=+234.878909172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-726sd" (UID: "6e970a71-b45e-4fdc-a726-6a1e3b97e91c") : secret "cluster-monitoring-operator-tls" not found Sep 13 19:08:02.998209 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.998159 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-trusted-ca\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.998578 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.998518 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:02.998971 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.998949 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-certificates\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:02.999172 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:02.999097 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-ca-trust-extracted\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:03.000183 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.000159 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-image-registry-private-configuration\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:03.000292 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.000262 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-installation-pull-secrets\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:03.007751 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.007427 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-bound-sa-token\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:03.007904 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.007847 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szt7z\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-kube-api-access-szt7z\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:03.008223 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.008196 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgksk\" (UniqueName: \"kubernetes.io/projected/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-kube-api-access-kgksk\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:03.008607 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.008582 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsrcb\" (UniqueName: \"kubernetes.io/projected/b4d96614-aac8-4a7e-86b1-007714a7a27f-kube-api-access-hsrcb\") pod \"network-check-source-6f8b768455-q48gb\" (UID: \"b4d96614-aac8-4a7e-86b1-007714a7a27f\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-q48gb" Sep 13 19:08:03.073560 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.073526 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-q48gb" Sep 13 19:08:03.252656 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.252567 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf"] Sep 13 19:08:03.254668 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.254644 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-875dw"] Sep 13 19:08:03.256081 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:03.256057 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc5e33f3_8e38_43dc_9f04_bef15c0d860c.slice/crio-d40df6872eeef3733995abfbca5f02f837ca73f006a186f8e51f9f388bc6f12a WatchSource:0}: Error finding container d40df6872eeef3733995abfbca5f02f837ca73f006a186f8e51f9f388bc6f12a: Status 404 returned error can't find the container with id d40df6872eeef3733995abfbca5f02f837ca73f006a186f8e51f9f388bc6f12a Sep 13 19:08:03.257205 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:03.257180 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod892493a1_d1c1_4f71_9621_77beb61a4a95.slice/crio-5e960c65bcac6b1da3b9886e625a69b1a4be1a54ad05ca460e94cf1750eaf157 WatchSource:0}: Error finding container 5e960c65bcac6b1da3b9886e625a69b1a4be1a54ad05ca460e94cf1750eaf157: Status 404 returned error can't find the container with id 5e960c65bcac6b1da3b9886e625a69b1a4be1a54ad05ca460e94cf1750eaf157 Sep 13 19:08:03.341270 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.341209 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-q48gb"] Sep 13 19:08:03.343910 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:03.343882 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4d96614_aac8_4a7e_86b1_007714a7a27f.slice/crio-0ee055984d86691ae0d6738557e974ac070fc0770fe66814aed9a6e1e636fd60 WatchSource:0}: Error finding container 0ee055984d86691ae0d6738557e974ac070fc0770fe66814aed9a6e1e636fd60: Status 404 returned error can't find the container with id 0ee055984d86691ae0d6738557e974ac070fc0770fe66814aed9a6e1e636fd60 Sep 13 19:08:03.501137 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.501091 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:03.501349 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.501172 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:03.501349 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:03.501267 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 19:08:03.501349 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:03.501293 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59698f7bd5-xn44h: secret "image-registry-tls" not found Sep 13 19:08:03.501349 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:03.501326 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 13 19:08:03.501484 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:03.501366 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls podName:7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48 nodeName:}" failed. No retries permitted until 2026-09-13 19:08:04.5013447 +0000 UTC m=+235.882873715 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls") pod "image-registry-59698f7bd5-xn44h" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48") : secret "image-registry-tls" not found Sep 13 19:08:03.501484 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:03.501380 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls podName:6e970a71-b45e-4fdc-a726-6a1e3b97e91c nodeName:}" failed. No retries permitted until 2026-09-13 19:08:04.501374186 +0000 UTC m=+235.882903192 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-726sd" (UID: "6e970a71-b45e-4fdc-a726-6a1e3b97e91c") : secret "cluster-monitoring-operator-tls" not found Sep 13 19:08:03.936997 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.936700 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-q48gb" event={"ID":"b4d96614-aac8-4a7e-86b1-007714a7a27f","Type":"ContainerStarted","Data":"3be5ce3413f6462560a73bc15ec36a912c20cc1587486f0e5fde6148e5213ba3"} Sep 13 19:08:03.936997 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.936764 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-q48gb" event={"ID":"b4d96614-aac8-4a7e-86b1-007714a7a27f","Type":"ContainerStarted","Data":"0ee055984d86691ae0d6738557e974ac070fc0770fe66814aed9a6e1e636fd60"} Sep 13 19:08:03.938305 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.938267 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-875dw" event={"ID":"892493a1-d1c1-4f71-9621-77beb61a4a95","Type":"ContainerStarted","Data":"5e960c65bcac6b1da3b9886e625a69b1a4be1a54ad05ca460e94cf1750eaf157"} Sep 13 19:08:03.939698 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.939666 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" event={"ID":"cc5e33f3-8e38-43dc-9f04-bef15c0d860c","Type":"ContainerStarted","Data":"d40df6872eeef3733995abfbca5f02f837ca73f006a186f8e51f9f388bc6f12a"} Sep 13 19:08:03.952149 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:03.952084 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-q48gb" podStartSLOduration=1.952061317 podStartE2EDuration="1.952061317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:03.951110405 +0000 UTC m=+235.332639430" watchObservedRunningTime="2026-09-13 19:08:03.952061317 +0000 UTC m=+235.333590348" Sep 13 19:08:04.510632 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:04.510581 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:04.510824 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:04.510652 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:04.510824 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:04.510738 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 13 19:08:04.510824 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:04.510806 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls podName:6e970a71-b45e-4fdc-a726-6a1e3b97e91c nodeName:}" failed. No retries permitted until 2026-09-13 19:08:06.510791017 +0000 UTC m=+237.892320023 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-726sd" (UID: "6e970a71-b45e-4fdc-a726-6a1e3b97e91c") : secret "cluster-monitoring-operator-tls" not found Sep 13 19:08:04.511004 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:04.510750 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 19:08:04.511004 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:04.510848 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59698f7bd5-xn44h: secret "image-registry-tls" not found Sep 13 19:08:04.511004 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:04.510896 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls podName:7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48 nodeName:}" failed. No retries permitted until 2026-09-13 19:08:06.51088236 +0000 UTC m=+237.892411373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls") pod "image-registry-59698f7bd5-xn44h" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48") : secret "image-registry-tls" not found Sep 13 19:08:06.527095 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.527054 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:06.527509 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.527143 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:06.527509 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:06.527217 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 13 19:08:06.527509 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:06.527292 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls podName:6e970a71-b45e-4fdc-a726-6a1e3b97e91c nodeName:}" failed. No retries permitted until 2026-09-13 19:08:10.52727802 +0000 UTC m=+241.908807031 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-726sd" (UID: "6e970a71-b45e-4fdc-a726-6a1e3b97e91c") : secret "cluster-monitoring-operator-tls" not found Sep 13 19:08:06.527509 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:06.527312 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 19:08:06.527509 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:06.527331 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59698f7bd5-xn44h: secret "image-registry-tls" not found Sep 13 19:08:06.527509 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:06.527371 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls podName:7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48 nodeName:}" failed. No retries permitted until 2026-09-13 19:08:10.52735765 +0000 UTC m=+241.908886661 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls") pod "image-registry-59698f7bd5-xn44h" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48") : secret "image-registry-tls" not found Sep 13 19:08:06.770353 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.770320 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lkmtm"] Sep 13 19:08:06.772491 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.772474 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:06.774879 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.774853 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 13 19:08:06.775221 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.775092 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 13 19:08:06.776014 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.775996 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 13 19:08:06.776117 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.776001 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9flm9\"" Sep 13 19:08:06.776117 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.776061 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 13 19:08:06.782698 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.782649 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lkmtm"] Sep 13 19:08:06.931372 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.931334 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/64b72404-61ee-45b3-bfcd-fcf8a166664b-data-volume\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:06.931372 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.931379 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:06.931629 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.931411 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/64b72404-61ee-45b3-bfcd-fcf8a166664b-crio-socket\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:06.931629 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.931519 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxfhn\" (UniqueName: \"kubernetes.io/projected/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-api-access-qxfhn\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:06.931629 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.931618 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:06.950805 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.950779 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/0.log" Sep 13 19:08:06.950940 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.950819 2563 generic.go:356] "Generic (PLEG): container finished" podID="892493a1-d1c1-4f71-9621-77beb61a4a95" containerID="649d467e72fe877ca019c1c154c0e12b29303e8e9aa9396249b743f5d0a8f158" exitCode=255 Sep 13 19:08:06.950940 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.950853 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-875dw" event={"ID":"892493a1-d1c1-4f71-9621-77beb61a4a95","Type":"ContainerDied","Data":"649d467e72fe877ca019c1c154c0e12b29303e8e9aa9396249b743f5d0a8f158"} Sep 13 19:08:06.951182 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.951129 2563 scope.go:117] "RemoveContainer" containerID="649d467e72fe877ca019c1c154c0e12b29303e8e9aa9396249b743f5d0a8f158" Sep 13 19:08:06.952474 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.952453 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" event={"ID":"cc5e33f3-8e38-43dc-9f04-bef15c0d860c","Type":"ContainerStarted","Data":"3f5aa5ad02ac4c894a037d472f3b55512488754617311f79867f7540db35c864"} Sep 13 19:08:06.981904 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:06.981847 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" podStartSLOduration=2.293813256 podStartE2EDuration="4.981828876s" totalImagesPullingTime="2.68801562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:06.980487922 +0000 UTC m=+238.362016951" watchObservedRunningTime="2026-09-13 19:08:06.981828876 +0000 UTC m=+238.363357904" Sep 13 19:08:07.033356 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.033267 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:07.033356 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.033333 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/64b72404-61ee-45b3-bfcd-fcf8a166664b-crio-socket\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:07.033569 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:07.033384 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 13 19:08:07.033569 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:07.033457 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls podName:64b72404-61ee-45b3-bfcd-fcf8a166664b nodeName:}" failed. No retries permitted until 2026-09-13 19:08:07.533437041 +0000 UTC m=+238.914966062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lkmtm" (UID: "64b72404-61ee-45b3-bfcd-fcf8a166664b") : secret "insights-runtime-extractor-tls" not found Sep 13 19:08:07.033569 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.033511 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/64b72404-61ee-45b3-bfcd-fcf8a166664b-crio-socket\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:07.033569 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.033396 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxfhn\" (UniqueName: \"kubernetes.io/projected/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-api-access-qxfhn\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:07.033768 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.033729 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:07.034165 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:07.033924 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 13 19:08:07.034165 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:07.033990 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm podName:64b72404-61ee-45b3-bfcd-fcf8a166664b nodeName:}" failed. No retries permitted until 2026-09-13 19:08:07.533973557 +0000 UTC m=+238.915502581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lkmtm" (UID: "64b72404-61ee-45b3-bfcd-fcf8a166664b") : configmap "kube-rbac-proxy" not found Sep 13 19:08:07.034165 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.034037 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/64b72404-61ee-45b3-bfcd-fcf8a166664b-data-volume\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:07.034442 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.034421 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/64b72404-61ee-45b3-bfcd-fcf8a166664b-data-volume\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:07.042615 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.042580 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxfhn\" (UniqueName: \"kubernetes.io/projected/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-api-access-qxfhn\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:07.538514 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.538470 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:07.538870 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.538557 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:07.538870 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:07.538596 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 13 19:08:07.538870 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:07.538615 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 13 19:08:07.538870 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:07.538659 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm podName:64b72404-61ee-45b3-bfcd-fcf8a166664b nodeName:}" failed. No retries permitted until 2026-09-13 19:08:08.53864425 +0000 UTC m=+239.920173271 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lkmtm" (UID: "64b72404-61ee-45b3-bfcd-fcf8a166664b") : configmap "kube-rbac-proxy" not found Sep 13 19:08:07.538870 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:07.538674 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls podName:64b72404-61ee-45b3-bfcd-fcf8a166664b nodeName:}" failed. No retries permitted until 2026-09-13 19:08:08.538667717 +0000 UTC m=+239.920196723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lkmtm" (UID: "64b72404-61ee-45b3-bfcd-fcf8a166664b") : secret "insights-runtime-extractor-tls" not found Sep 13 19:08:07.957972 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.957897 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/1.log" Sep 13 19:08:07.958375 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.958359 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/0.log" Sep 13 19:08:07.958476 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.958399 2563 generic.go:356] "Generic (PLEG): container finished" podID="892493a1-d1c1-4f71-9621-77beb61a4a95" containerID="f397dbf5435e884bfff05acb605d5d2d627e16b57cb7f5f536b8ef4618395d38" exitCode=255 Sep 13 19:08:07.958529 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.958481 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-875dw" event={"ID":"892493a1-d1c1-4f71-9621-77beb61a4a95","Type":"ContainerDied","Data":"f397dbf5435e884bfff05acb605d5d2d627e16b57cb7f5f536b8ef4618395d38"} Sep 13 19:08:07.958566 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.958540 2563 scope.go:117] "RemoveContainer" containerID="649d467e72fe877ca019c1c154c0e12b29303e8e9aa9396249b743f5d0a8f158" Sep 13 19:08:07.958757 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:07.958742 2563 scope.go:117] "RemoveContainer" containerID="f397dbf5435e884bfff05acb605d5d2d627e16b57cb7f5f536b8ef4618395d38" Sep 13 19:08:07.958932 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:07.958912 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-875dw_openshift-console-operator(892493a1-d1c1-4f71-9621-77beb61a4a95)\"" pod="openshift-console-operator/console-operator-696884fb9c-875dw" podUID="892493a1-d1c1-4f71-9621-77beb61a4a95" Sep 13 19:08:08.546747 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.546712 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:08.547135 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.546757 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:08.547135 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:08.546852 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 13 19:08:08.547135 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:08.546925 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm podName:64b72404-61ee-45b3-bfcd-fcf8a166664b nodeName:}" failed. No retries permitted until 2026-09-13 19:08:10.546909237 +0000 UTC m=+241.928438249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lkmtm" (UID: "64b72404-61ee-45b3-bfcd-fcf8a166664b") : configmap "kube-rbac-proxy" not found Sep 13 19:08:08.547135 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:08.546863 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 13 19:08:08.547135 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:08.546995 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls podName:64b72404-61ee-45b3-bfcd-fcf8a166664b nodeName:}" failed. No retries permitted until 2026-09-13 19:08:10.546983938 +0000 UTC m=+241.928512949 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lkmtm" (UID: "64b72404-61ee-45b3-bfcd-fcf8a166664b") : secret "insights-runtime-extractor-tls" not found Sep 13 19:08:08.915123 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.915045 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-xdjqq"] Sep 13 19:08:08.917802 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.917784 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" Sep 13 19:08:08.920600 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.920577 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 13 19:08:08.920731 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.920621 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 13 19:08:08.920731 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.920665 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 13 19:08:08.921579 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.921552 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-p5lcd\"" Sep 13 19:08:08.921579 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.921559 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 13 19:08:08.926499 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.926473 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-xdjqq"] Sep 13 19:08:08.933228 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.933210 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ttddh_31fa23e6-3a87-4f8f-9f54-48853123eb54/dns-node-resolver/0.log" Sep 13 19:08:08.963396 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.963369 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/1.log" Sep 13 19:08:08.963702 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:08.963688 2563 scope.go:117] "RemoveContainer" containerID="f397dbf5435e884bfff05acb605d5d2d627e16b57cb7f5f536b8ef4618395d38" Sep 13 19:08:08.963857 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:08.963839 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-875dw_openshift-console-operator(892493a1-d1c1-4f71-9621-77beb61a4a95)\"" pod="openshift-console-operator/console-operator-696884fb9c-875dw" podUID="892493a1-d1c1-4f71-9621-77beb61a4a95" Sep 13 19:08:09.050685 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.050655 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/029a7465-4ba7-4f92-adfb-614e511bf105-signing-cabundle\") pod \"service-ca-568869fdc5-xdjqq\" (UID: \"029a7465-4ba7-4f92-adfb-614e511bf105\") " pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" Sep 13 19:08:09.050685 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.050689 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/029a7465-4ba7-4f92-adfb-614e511bf105-signing-key\") pod \"service-ca-568869fdc5-xdjqq\" (UID: \"029a7465-4ba7-4f92-adfb-614e511bf105\") " pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" Sep 13 19:08:09.050883 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.050812 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpsrg\" (UniqueName: \"kubernetes.io/projected/029a7465-4ba7-4f92-adfb-614e511bf105-kube-api-access-fpsrg\") pod \"service-ca-568869fdc5-xdjqq\" (UID: \"029a7465-4ba7-4f92-adfb-614e511bf105\") " pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" Sep 13 19:08:09.151395 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.151355 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpsrg\" (UniqueName: \"kubernetes.io/projected/029a7465-4ba7-4f92-adfb-614e511bf105-kube-api-access-fpsrg\") pod \"service-ca-568869fdc5-xdjqq\" (UID: \"029a7465-4ba7-4f92-adfb-614e511bf105\") " pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" Sep 13 19:08:09.151599 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.151450 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/029a7465-4ba7-4f92-adfb-614e511bf105-signing-cabundle\") pod \"service-ca-568869fdc5-xdjqq\" (UID: \"029a7465-4ba7-4f92-adfb-614e511bf105\") " pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" Sep 13 19:08:09.151599 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.151477 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/029a7465-4ba7-4f92-adfb-614e511bf105-signing-key\") pod \"service-ca-568869fdc5-xdjqq\" (UID: \"029a7465-4ba7-4f92-adfb-614e511bf105\") " pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" Sep 13 19:08:09.153567 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.153526 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 13 19:08:09.153684 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.153670 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 13 19:08:09.158208 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.158191 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 13 19:08:09.162461 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.162439 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/029a7465-4ba7-4f92-adfb-614e511bf105-signing-cabundle\") pod \"service-ca-568869fdc5-xdjqq\" (UID: \"029a7465-4ba7-4f92-adfb-614e511bf105\") " pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" Sep 13 19:08:09.164249 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.164206 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/029a7465-4ba7-4f92-adfb-614e511bf105-signing-key\") pod \"service-ca-568869fdc5-xdjqq\" (UID: \"029a7465-4ba7-4f92-adfb-614e511bf105\") " pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" Sep 13 19:08:09.168922 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.168872 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 13 19:08:09.180109 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.180083 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpsrg\" (UniqueName: \"kubernetes.io/projected/029a7465-4ba7-4f92-adfb-614e511bf105-kube-api-access-fpsrg\") pod \"service-ca-568869fdc5-xdjqq\" (UID: \"029a7465-4ba7-4f92-adfb-614e511bf105\") " pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" Sep 13 19:08:09.228457 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.228425 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-p5lcd\"" Sep 13 19:08:09.237246 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.237214 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" Sep 13 19:08:09.503597 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.503563 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-xdjqq"] Sep 13 19:08:09.507422 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:09.507394 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029a7465_4ba7_4f92_adfb_614e511bf105.slice/crio-90449b521da1477922a0fe45bd74cf18eb03aac57dcac8dd6b2256ce42612070 WatchSource:0}: Error finding container 90449b521da1477922a0fe45bd74cf18eb03aac57dcac8dd6b2256ce42612070: Status 404 returned error can't find the container with id 90449b521da1477922a0fe45bd74cf18eb03aac57dcac8dd6b2256ce42612070 Sep 13 19:08:09.532174 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.532148 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fbmxs_effec463-bf64-4a1c-b7ad-9bee7e4901df/node-ca/0.log" Sep 13 19:08:09.967784 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:09.967751 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" event={"ID":"029a7465-4ba7-4f92-adfb-614e511bf105","Type":"ContainerStarted","Data":"90449b521da1477922a0fe45bd74cf18eb03aac57dcac8dd6b2256ce42612070"} Sep 13 19:08:10.332756 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:10.332675 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4zfmc_bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327/serve-healthcheck-canary/0.log" Sep 13 19:08:10.564411 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:10.564368 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:10.564655 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:10.564428 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:10.564655 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:10.564470 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:10.564655 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:10.564465 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 13 19:08:10.564655 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:10.564568 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm podName:64b72404-61ee-45b3-bfcd-fcf8a166664b nodeName:}" failed. No retries permitted until 2026-09-13 19:08:14.564552787 +0000 UTC m=+245.946081794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lkmtm" (UID: "64b72404-61ee-45b3-bfcd-fcf8a166664b") : configmap "kube-rbac-proxy" not found Sep 13 19:08:10.564655 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:10.564567 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 13 19:08:10.564655 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:10.564593 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 13 19:08:10.564655 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:10.564598 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls podName:64b72404-61ee-45b3-bfcd-fcf8a166664b nodeName:}" failed. No retries permitted until 2026-09-13 19:08:14.564592244 +0000 UTC m=+245.946121251 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lkmtm" (UID: "64b72404-61ee-45b3-bfcd-fcf8a166664b") : secret "insights-runtime-extractor-tls" not found Sep 13 19:08:10.564982 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:10.564669 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:10.564982 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:10.564735 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls podName:6e970a71-b45e-4fdc-a726-6a1e3b97e91c nodeName:}" failed. No retries permitted until 2026-09-13 19:08:18.564716437 +0000 UTC m=+249.946245656 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-726sd" (UID: "6e970a71-b45e-4fdc-a726-6a1e3b97e91c") : secret "cluster-monitoring-operator-tls" not found Sep 13 19:08:10.564982 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:10.564738 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 19:08:10.564982 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:10.564751 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59698f7bd5-xn44h: secret "image-registry-tls" not found Sep 13 19:08:10.564982 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:10.564794 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls podName:7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48 nodeName:}" failed. No retries permitted until 2026-09-13 19:08:18.564780843 +0000 UTC m=+249.946309853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls") pod "image-registry-59698f7bd5-xn44h" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48") : secret "image-registry-tls" not found Sep 13 19:08:11.975254 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:11.975201 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" event={"ID":"029a7465-4ba7-4f92-adfb-614e511bf105","Type":"ContainerStarted","Data":"f8693e4a5cfadca49733f3e9abac38e49d0ec7b901e72b694e3ae5a67a6cb84e"} Sep 13 19:08:11.989536 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:11.989481 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-xdjqq" podStartSLOduration=2.071004182 podStartE2EDuration="3.98946478s" totalImagesPullingTime="1.918460598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:11.98864947 +0000 UTC m=+243.370178500" watchObservedRunningTime="2026-09-13 19:08:11.98946478 +0000 UTC m=+243.370993808" Sep 13 19:08:12.983812 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:12.983777 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:12.984299 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:12.983835 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:12.984299 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:12.984204 2563 scope.go:117] "RemoveContainer" containerID="f397dbf5435e884bfff05acb605d5d2d627e16b57cb7f5f536b8ef4618395d38" Sep 13 19:08:12.984418 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:12.984392 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-875dw_openshift-console-operator(892493a1-d1c1-4f71-9621-77beb61a4a95)\"" pod="openshift-console-operator/console-operator-696884fb9c-875dw" podUID="892493a1-d1c1-4f71-9621-77beb61a4a95" Sep 13 19:08:14.600041 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:14.599996 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:14.600041 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:14.600049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:14.600520 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:14.600127 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 13 19:08:14.600520 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:14.600186 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 13 19:08:14.600520 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:14.600197 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm podName:64b72404-61ee-45b3-bfcd-fcf8a166664b nodeName:}" failed. No retries permitted until 2026-09-13 19:08:22.600181483 +0000 UTC m=+253.981710490 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lkmtm" (UID: "64b72404-61ee-45b3-bfcd-fcf8a166664b") : configmap "kube-rbac-proxy" not found Sep 13 19:08:14.600520 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:14.600233 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls podName:64b72404-61ee-45b3-bfcd-fcf8a166664b nodeName:}" failed. No retries permitted until 2026-09-13 19:08:22.600222707 +0000 UTC m=+253.981751714 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lkmtm" (UID: "64b72404-61ee-45b3-bfcd-fcf8a166664b") : secret "insights-runtime-extractor-tls" not found Sep 13 19:08:18.633015 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:18.632948 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:18.633483 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:18.633048 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:18.633483 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:18.633109 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 13 19:08:18.633483 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:18.633178 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls podName:6e970a71-b45e-4fdc-a726-6a1e3b97e91c nodeName:}" failed. No retries permitted until 2026-09-13 19:08:34.633161773 +0000 UTC m=+266.014690785 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-726sd" (UID: "6e970a71-b45e-4fdc-a726-6a1e3b97e91c") : secret "cluster-monitoring-operator-tls" not found Sep 13 19:08:18.635593 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:18.635560 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls\") pod \"image-registry-59698f7bd5-xn44h\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:18.683762 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:18.683728 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mg4q7\"" Sep 13 19:08:18.692654 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:18.692617 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:18.963151 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:18.963047 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59698f7bd5-xn44h"] Sep 13 19:08:18.967052 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:18.967017 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eb4d4e7_a4ca_4bc6_a00a_c752f8c15c48.slice/crio-568e6b15529b4715964b80f3b5d9d15a0510d38cbd48cce1cd0763d597ff633a WatchSource:0}: Error finding container 568e6b15529b4715964b80f3b5d9d15a0510d38cbd48cce1cd0763d597ff633a: Status 404 returned error can't find the container with id 568e6b15529b4715964b80f3b5d9d15a0510d38cbd48cce1cd0763d597ff633a Sep 13 19:08:18.999898 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:18.999857 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" event={"ID":"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48","Type":"ContainerStarted","Data":"568e6b15529b4715964b80f3b5d9d15a0510d38cbd48cce1cd0763d597ff633a"} Sep 13 19:08:20.004705 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:20.004673 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" event={"ID":"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48","Type":"ContainerStarted","Data":"fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e"} Sep 13 19:08:20.005087 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:20.004896 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:20.022205 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:20.022146 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" podStartSLOduration=18.022129211 podStartE2EDuration="18.022129211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:20.021215502 +0000 UTC m=+251.402744534" watchObservedRunningTime="2026-09-13 19:08:20.022129211 +0000 UTC m=+251.403658239" Sep 13 19:08:20.953358 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:20.953308 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:08:20.955957 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:20.955928 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526ccac7-e62a-45c1-bc16-1c2f1bc7aca5-metrics-certs\") pod \"network-metrics-daemon-2wc2z\" (UID: \"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5\") " pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:08:21.227505 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:21.227475 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-454nd\"" Sep 13 19:08:21.235312 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:21.235284 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wc2z" Sep 13 19:08:21.503947 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:21.503921 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2wc2z"] Sep 13 19:08:21.506345 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:21.506313 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod526ccac7_e62a_45c1_bc16_1c2f1bc7aca5.slice/crio-1e9e55afa4eea04757b8b0b1ebbd33726a4e9362fe0fd3ace5a7c2a4d4eae45a WatchSource:0}: Error finding container 1e9e55afa4eea04757b8b0b1ebbd33726a4e9362fe0fd3ace5a7c2a4d4eae45a: Status 404 returned error can't find the container with id 1e9e55afa4eea04757b8b0b1ebbd33726a4e9362fe0fd3ace5a7c2a4d4eae45a Sep 13 19:08:22.013283 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:22.013223 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2wc2z" event={"ID":"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5","Type":"ContainerStarted","Data":"1e9e55afa4eea04757b8b0b1ebbd33726a4e9362fe0fd3ace5a7c2a4d4eae45a"} Sep 13 19:08:22.667547 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:22.667495 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:22.667547 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:22.667553 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:22.667970 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:22.667672 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 13 19:08:22.667970 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:22.667726 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls podName:64b72404-61ee-45b3-bfcd-fcf8a166664b nodeName:}" failed. No retries permitted until 2026-09-13 19:08:38.667711982 +0000 UTC m=+270.049240989 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lkmtm" (UID: "64b72404-61ee-45b3-bfcd-fcf8a166664b") : secret "insights-runtime-extractor-tls" not found Sep 13 19:08:22.668091 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:22.668069 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64b72404-61ee-45b3-bfcd-fcf8a166664b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:23.018690 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:23.018658 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2wc2z" event={"ID":"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5","Type":"ContainerStarted","Data":"51ce452fe67c216183c6cb99df18b9b97b104f02b3706028e37ccd6aa839db3f"} Sep 13 19:08:23.018860 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:23.018709 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2wc2z" event={"ID":"526ccac7-e62a-45c1-bc16-1c2f1bc7aca5","Type":"ContainerStarted","Data":"252579da142ad79f4fafdbf121ae171d4941665e878edfc8fb701d9983355fe7"} Sep 13 19:08:23.034077 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:23.034023 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2wc2z" podStartSLOduration=253.092585568 podStartE2EDuration="4m14.034005314s" totalImagesPullingTime="941.419746ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:23.032733092 +0000 UTC m=+254.414262123" watchObservedRunningTime="2026-09-13 19:08:23.034005314 +0000 UTC m=+254.415534408" Sep 13 19:08:27.124852 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.124821 2563 scope.go:117] "RemoveContainer" containerID="f397dbf5435e884bfff05acb605d5d2d627e16b57cb7f5f536b8ef4618395d38" Sep 13 19:08:27.709288 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.709234 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59698f7bd5-xn44h"] Sep 13 19:08:27.742412 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.742374 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5569d8f464-5dzh9"] Sep 13 19:08:27.744410 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.744380 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.758098 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.758063 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5569d8f464-5dzh9"] Sep 13 19:08:27.808906 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.808861 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c10e26c-528d-4f49-806d-9c15d418e11c-image-registry-private-configuration\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.809201 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.809150 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c10e26c-528d-4f49-806d-9c15d418e11c-bound-sa-token\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.809201 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.809184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c10e26c-528d-4f49-806d-9c15d418e11c-installation-pull-secrets\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.809369 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.809302 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c10e26c-528d-4f49-806d-9c15d418e11c-ca-trust-extracted\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.809369 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.809323 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c10e26c-528d-4f49-806d-9c15d418e11c-trusted-ca\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.809369 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.809349 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh2cr\" (UniqueName: \"kubernetes.io/projected/1c10e26c-528d-4f49-806d-9c15d418e11c-kube-api-access-dh2cr\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.809485 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.809394 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c10e26c-528d-4f49-806d-9c15d418e11c-registry-tls\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.809485 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.809434 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c10e26c-528d-4f49-806d-9c15d418e11c-registry-certificates\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.910104 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.910061 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c10e26c-528d-4f49-806d-9c15d418e11c-image-registry-private-configuration\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.910376 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.910120 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c10e26c-528d-4f49-806d-9c15d418e11c-bound-sa-token\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.910376 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.910144 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c10e26c-528d-4f49-806d-9c15d418e11c-installation-pull-secrets\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.910376 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.910206 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c10e26c-528d-4f49-806d-9c15d418e11c-ca-trust-extracted\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.910376 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.910222 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c10e26c-528d-4f49-806d-9c15d418e11c-trusted-ca\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.910376 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.910259 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh2cr\" (UniqueName: \"kubernetes.io/projected/1c10e26c-528d-4f49-806d-9c15d418e11c-kube-api-access-dh2cr\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.910376 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.910285 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c10e26c-528d-4f49-806d-9c15d418e11c-registry-tls\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.910376 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.910301 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c10e26c-528d-4f49-806d-9c15d418e11c-registry-certificates\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.911170 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.911142 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c10e26c-528d-4f49-806d-9c15d418e11c-registry-certificates\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.911524 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.911495 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c10e26c-528d-4f49-806d-9c15d418e11c-trusted-ca\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.911808 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.911784 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c10e26c-528d-4f49-806d-9c15d418e11c-ca-trust-extracted\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.913363 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.913318 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c10e26c-528d-4f49-806d-9c15d418e11c-image-registry-private-configuration\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.913646 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.913621 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c10e26c-528d-4f49-806d-9c15d418e11c-installation-pull-secrets\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.914353 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.914335 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c10e26c-528d-4f49-806d-9c15d418e11c-registry-tls\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.929440 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.929410 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c10e26c-528d-4f49-806d-9c15d418e11c-bound-sa-token\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.943176 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.943140 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh2cr\" (UniqueName: \"kubernetes.io/projected/1c10e26c-528d-4f49-806d-9c15d418e11c-kube-api-access-dh2cr\") pod \"image-registry-5569d8f464-5dzh9\" (UID: \"1c10e26c-528d-4f49-806d-9c15d418e11c\") " pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:27.986652 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.986616 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-sx5m5"] Sep 13 19:08:27.989690 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.989667 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-sx5m5" Sep 13 19:08:27.992098 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.992075 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 13 19:08:27.992229 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.992208 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-hhgnq\"" Sep 13 19:08:27.992318 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:27.992215 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 13 19:08:28.001868 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.001840 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-sx5m5"] Sep 13 19:08:28.011005 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.010973 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgjbm\" (UniqueName: \"kubernetes.io/projected/066e97f0-a26a-4c7a-a65f-ac4cdf7ca8c0-kube-api-access-dgjbm\") pod \"downloads-76f99584f4-sx5m5\" (UID: \"066e97f0-a26a-4c7a-a65f-ac4cdf7ca8c0\") " pod="openshift-console/downloads-76f99584f4-sx5m5" Sep 13 19:08:28.038599 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.038566 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/1.log" Sep 13 19:08:28.038792 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.038657 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-875dw" event={"ID":"892493a1-d1c1-4f71-9621-77beb61a4a95","Type":"ContainerStarted","Data":"e7e4743b134ba791f046e649f3aecf014bb8e6c756b4f8f6e245613e54b59063"} Sep 13 19:08:28.038940 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.038918 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:28.043678 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.043649 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-875dw" Sep 13 19:08:28.054097 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.054064 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:28.055788 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.055744 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-875dw" podStartSLOduration=23.370551632 podStartE2EDuration="26.055727427s" totalImagesPullingTime="2.685175795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:28.054568904 +0000 UTC m=+259.436097934" watchObservedRunningTime="2026-09-13 19:08:28.055727427 +0000 UTC m=+259.437256456" Sep 13 19:08:28.112000 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.111957 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgjbm\" (UniqueName: \"kubernetes.io/projected/066e97f0-a26a-4c7a-a65f-ac4cdf7ca8c0-kube-api-access-dgjbm\") pod \"downloads-76f99584f4-sx5m5\" (UID: \"066e97f0-a26a-4c7a-a65f-ac4cdf7ca8c0\") " pod="openshift-console/downloads-76f99584f4-sx5m5" Sep 13 19:08:28.125661 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.125620 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgjbm\" (UniqueName: \"kubernetes.io/projected/066e97f0-a26a-4c7a-a65f-ac4cdf7ca8c0-kube-api-access-dgjbm\") pod \"downloads-76f99584f4-sx5m5\" (UID: \"066e97f0-a26a-4c7a-a65f-ac4cdf7ca8c0\") " pod="openshift-console/downloads-76f99584f4-sx5m5" Sep 13 19:08:28.299571 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.299455 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-sx5m5" Sep 13 19:08:28.329463 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.329310 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5569d8f464-5dzh9"] Sep 13 19:08:28.332008 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:28.331978 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c10e26c_528d_4f49_806d_9c15d418e11c.slice/crio-16d686bdd107181b728017f3ecb34556df0d08f87f3254f73b081823a1abcbf6 WatchSource:0}: Error finding container 16d686bdd107181b728017f3ecb34556df0d08f87f3254f73b081823a1abcbf6: Status 404 returned error can't find the container with id 16d686bdd107181b728017f3ecb34556df0d08f87f3254f73b081823a1abcbf6 Sep 13 19:08:28.576430 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:28.576203 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-sx5m5"] Sep 13 19:08:28.578713 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:28.578685 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod066e97f0_a26a_4c7a_a65f_ac4cdf7ca8c0.slice/crio-01f0972d9ef9973cda6675880d48179015eb0cb220409c00fb61b032269a312f WatchSource:0}: Error finding container 01f0972d9ef9973cda6675880d48179015eb0cb220409c00fb61b032269a312f: Status 404 returned error can't find the container with id 01f0972d9ef9973cda6675880d48179015eb0cb220409c00fb61b032269a312f Sep 13 19:08:29.045113 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:29.045067 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" event={"ID":"1c10e26c-528d-4f49-806d-9c15d418e11c","Type":"ContainerStarted","Data":"bcc2e5678f9e74258b6bf55fa6af43af14511ed547f96ee7e6e6951a8091b8d2"} Sep 13 19:08:29.045325 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:29.045139 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" event={"ID":"1c10e26c-528d-4f49-806d-9c15d418e11c","Type":"ContainerStarted","Data":"16d686bdd107181b728017f3ecb34556df0d08f87f3254f73b081823a1abcbf6"} Sep 13 19:08:29.045325 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:29.045187 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:29.046697 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:29.046660 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-sx5m5" event={"ID":"066e97f0-a26a-4c7a-a65f-ac4cdf7ca8c0","Type":"ContainerStarted","Data":"01f0972d9ef9973cda6675880d48179015eb0cb220409c00fb61b032269a312f"} Sep 13 19:08:29.066209 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:29.066149 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" podStartSLOduration=2.066128754 podStartE2EDuration="2.066128754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:08:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:29.06490109 +0000 UTC m=+260.446430129" watchObservedRunningTime="2026-09-13 19:08:29.066128754 +0000 UTC m=+260.447657785" Sep 13 19:08:34.670464 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:34.670364 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:34.673088 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:34.673060 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e970a71-b45e-4fdc-a726-6a1e3b97e91c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-726sd\" (UID: \"6e970a71-b45e-4fdc-a726-6a1e3b97e91c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:34.889744 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:34.889704 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-g767s\"" Sep 13 19:08:34.897729 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:34.897690 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" Sep 13 19:08:35.183530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:35.183494 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd"] Sep 13 19:08:35.186334 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:35.186297 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e970a71_b45e_4fdc_a726_6a1e3b97e91c.slice/crio-69391b0f41f76b6ecd155f6eaed1f9c541c7444c966bd460fdf07b676aa18427 WatchSource:0}: Error finding container 69391b0f41f76b6ecd155f6eaed1f9c541c7444c966bd460fdf07b676aa18427: Status 404 returned error can't find the container with id 69391b0f41f76b6ecd155f6eaed1f9c541c7444c966bd460fdf07b676aa18427 Sep 13 19:08:36.077806 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:36.077766 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" event={"ID":"6e970a71-b45e-4fdc-a726-6a1e3b97e91c","Type":"ContainerStarted","Data":"69391b0f41f76b6ecd155f6eaed1f9c541c7444c966bd460fdf07b676aa18427"} Sep 13 19:08:37.083467 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:37.083389 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" event={"ID":"6e970a71-b45e-4fdc-a726-6a1e3b97e91c","Type":"ContainerStarted","Data":"0cd3845cd281f59031ac254e9f99ad0583c3460f23d608cdd12b683d7fd60a58"} Sep 13 19:08:37.103519 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:37.103439 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-726sd" podStartSLOduration=33.344515946 podStartE2EDuration="35.103415461s" totalImagesPullingTime="1.758899515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:37.102927068 +0000 UTC m=+268.484456097" watchObservedRunningTime="2026-09-13 19:08:37.103415461 +0000 UTC m=+268.484944492" Sep 13 19:08:37.716631 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:37.716580 2563 patch_prober.go:28] interesting pod/image-registry-59698f7bd5-xn44h container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 13 19:08:37.716826 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:37.716651 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" podUID="7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 13 19:08:38.706353 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:38.706307 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:38.708919 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:38.708880 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64b72404-61ee-45b3-bfcd-fcf8a166664b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkmtm\" (UID: \"64b72404-61ee-45b3-bfcd-fcf8a166664b\") " pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:38.885256 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:38.885207 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9flm9\"" Sep 13 19:08:38.893333 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:38.893292 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lkmtm" Sep 13 19:08:44.832330 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.832292 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lfttb"] Sep 13 19:08:44.835904 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.835872 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:44.838775 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.838433 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 13 19:08:44.838775 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.838457 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 13 19:08:44.838775 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.838432 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dtclt\"" Sep 13 19:08:44.838775 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.838626 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 13 19:08:44.838775 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.838563 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 13 19:08:44.962373 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.962333 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-wtmp\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:44.962540 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.962387 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/191a7f56-c678-4efa-8f90-8e1d6016085b-metrics-client-ca\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:44.962540 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.962419 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/191a7f56-c678-4efa-8f90-8e1d6016085b-sys\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:44.962540 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.962472 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/191a7f56-c678-4efa-8f90-8e1d6016085b-root\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:44.962540 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.962522 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz276\" (UniqueName: \"kubernetes.io/projected/191a7f56-c678-4efa-8f90-8e1d6016085b-kube-api-access-fz276\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:44.962672 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.962566 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-textfile\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:44.962672 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.962591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-tls\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:44.962737 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.962660 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:44.962737 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:44.962699 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-accelerators-collector-config\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.063460 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.063399 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-tls\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.064551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.063474 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.064551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.063509 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-accelerators-collector-config\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.064551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.063608 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-wtmp\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.064551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.063641 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/191a7f56-c678-4efa-8f90-8e1d6016085b-metrics-client-ca\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.064551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.063672 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/191a7f56-c678-4efa-8f90-8e1d6016085b-sys\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.064551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.063697 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/191a7f56-c678-4efa-8f90-8e1d6016085b-root\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.064551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.063729 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz276\" (UniqueName: \"kubernetes.io/projected/191a7f56-c678-4efa-8f90-8e1d6016085b-kube-api-access-fz276\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.064551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.063761 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-textfile\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.064551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.064131 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-textfile\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.066474 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.066284 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/191a7f56-c678-4efa-8f90-8e1d6016085b-metrics-client-ca\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.066474 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.066435 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-wtmp\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.066664 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.066475 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/191a7f56-c678-4efa-8f90-8e1d6016085b-sys\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.066664 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.066490 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/191a7f56-c678-4efa-8f90-8e1d6016085b-root\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.066810 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.066789 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-accelerators-collector-config\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.067066 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.067045 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-tls\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.067985 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.067964 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/191a7f56-c678-4efa-8f90-8e1d6016085b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.075881 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.075857 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz276\" (UniqueName: \"kubernetes.io/projected/191a7f56-c678-4efa-8f90-8e1d6016085b-kube-api-access-fz276\") pod \"node-exporter-lfttb\" (UID: \"191a7f56-c678-4efa-8f90-8e1d6016085b\") " pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.115339 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.115217 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-sx5m5" event={"ID":"066e97f0-a26a-4c7a-a65f-ac4cdf7ca8c0","Type":"ContainerStarted","Data":"af772142ade4a4b88c5e2f92fad51ef952afeb0e999b51168e5ce3a83e0f5357"} Sep 13 19:08:45.115475 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.115441 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-sx5m5" Sep 13 19:08:45.116700 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.116675 2563 patch_prober.go:28] interesting pod/downloads-76f99584f4-sx5m5 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.17:8080/\": dial tcp 10.134.0.17:8080: connect: connection refused" start-of-body= Sep 13 19:08:45.116815 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.116718 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-76f99584f4-sx5m5" podUID="066e97f0-a26a-4c7a-a65f-ac4cdf7ca8c0" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.17:8080/\": dial tcp 10.134.0.17:8080: connect: connection refused" Sep 13 19:08:45.131513 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.131453 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-sx5m5" podStartSLOduration=1.7114006389999998 podStartE2EDuration="18.13143211s" totalImagesPullingTime="16.420031471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:08:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:45.131119221 +0000 UTC m=+276.512648251" watchObservedRunningTime="2026-09-13 19:08:45.13143211 +0000 UTC m=+276.512961140" Sep 13 19:08:45.149098 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.149063 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lfttb" Sep 13 19:08:45.158778 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:45.158742 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191a7f56_c678_4efa_8f90_8e1d6016085b.slice/crio-a45d1e2d899b140cf3ce90007e4df494c53401e0e32376bc97e34fa47062e96b WatchSource:0}: Error finding container a45d1e2d899b140cf3ce90007e4df494c53401e0e32376bc97e34fa47062e96b: Status 404 returned error can't find the container with id a45d1e2d899b140cf3ce90007e4df494c53401e0e32376bc97e34fa47062e96b Sep 13 19:08:45.193203 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.193158 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lkmtm"] Sep 13 19:08:45.197353 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:45.197297 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64b72404_61ee_45b3_bfcd_fcf8a166664b.slice/crio-72009f4043b7bd1c2f5eb9c306f531d635bbbad3a7f1f98271537c2ef0993670 WatchSource:0}: Error finding container 72009f4043b7bd1c2f5eb9c306f531d635bbbad3a7f1f98271537c2ef0993670: Status 404 returned error can't find the container with id 72009f4043b7bd1c2f5eb9c306f531d635bbbad3a7f1f98271537c2ef0993670 Sep 13 19:08:45.937314 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.935335 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 13 19:08:45.938823 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.938716 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:45.941663 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.941444 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 13 19:08:45.941663 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.941468 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 13 19:08:45.942335 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.941981 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dsqcj\"" Sep 13 19:08:45.942335 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.942178 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 13 19:08:45.942335 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.942316 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 13 19:08:45.944429 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.943415 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 13 19:08:45.944429 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.943637 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 13 19:08:45.944429 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.944011 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 13 19:08:45.944429 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.944266 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 13 19:08:45.944698 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.944463 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 13 19:08:45.957009 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:45.956975 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 13 19:08:46.076754 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.076661 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c3d16eb2-f983-4991-8d3f-a32e2e42a521-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.077026 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.076997 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.077230 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.077188 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.077409 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.077391 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c3d16eb2-f983-4991-8d3f-a32e2e42a521-config-out\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.077559 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.077541 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.077713 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.077696 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3d16eb2-f983-4991-8d3f-a32e2e42a521-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.077876 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.077860 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.078045 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.078027 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c3d16eb2-f983-4991-8d3f-a32e2e42a521-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.078211 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.078196 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-web-config\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.078407 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.078390 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3d16eb2-f983-4991-8d3f-a32e2e42a521-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.078558 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.078542 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qb4j\" (UniqueName: \"kubernetes.io/projected/c3d16eb2-f983-4991-8d3f-a32e2e42a521-kube-api-access-4qb4j\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.078740 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.078728 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-config-volume\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.078886 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.078869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.121431 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.121386 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lfttb" event={"ID":"191a7f56-c678-4efa-8f90-8e1d6016085b","Type":"ContainerStarted","Data":"a45d1e2d899b140cf3ce90007e4df494c53401e0e32376bc97e34fa47062e96b"} Sep 13 19:08:46.125974 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.124992 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lkmtm" event={"ID":"64b72404-61ee-45b3-bfcd-fcf8a166664b","Type":"ContainerStarted","Data":"fcee1a61449084e68db7852e3f88976b00898ec2e40e4c4d7f55fee661f85d79"} Sep 13 19:08:46.125974 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.125050 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lkmtm" event={"ID":"64b72404-61ee-45b3-bfcd-fcf8a166664b","Type":"ContainerStarted","Data":"dc994bb563e76d4b7cdfe96e27a77bf7eb087d8b85b4bcf2258b116cf25d50c2"} Sep 13 19:08:46.125974 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.125077 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lkmtm" event={"ID":"64b72404-61ee-45b3-bfcd-fcf8a166664b","Type":"ContainerStarted","Data":"72009f4043b7bd1c2f5eb9c306f531d635bbbad3a7f1f98271537c2ef0993670"} Sep 13 19:08:46.130856 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.130582 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-sx5m5" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179518 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-config-volume\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179569 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179609 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c3d16eb2-f983-4991-8d3f-a32e2e42a521-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179644 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179678 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179721 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c3d16eb2-f983-4991-8d3f-a32e2e42a521-config-out\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179749 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179775 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3d16eb2-f983-4991-8d3f-a32e2e42a521-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179806 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179829 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c3d16eb2-f983-4991-8d3f-a32e2e42a521-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179860 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-web-config\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179925 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3d16eb2-f983-4991-8d3f-a32e2e42a521-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.180828 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.179952 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qb4j\" (UniqueName: \"kubernetes.io/projected/c3d16eb2-f983-4991-8d3f-a32e2e42a521-kube-api-access-4qb4j\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.182212 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:46.182190 2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 13 19:08:46.184226 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.182426 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c3d16eb2-f983-4991-8d3f-a32e2e42a521-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.184226 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:46.182452 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-main-tls podName:c3d16eb2-f983-4991-8d3f-a32e2e42a521 nodeName:}" failed. No retries permitted until 2026-09-13 19:08:46.682411769 +0000 UTC m=+278.063940789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521") : secret "alertmanager-main-tls" not found Sep 13 19:08:46.184226 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.183443 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3d16eb2-f983-4991-8d3f-a32e2e42a521-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.186178 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.186154 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3d16eb2-f983-4991-8d3f-a32e2e42a521-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.189549 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.189485 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c3d16eb2-f983-4991-8d3f-a32e2e42a521-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.190658 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.190573 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.192839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.192790 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-web-config\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.193440 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.193414 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c3d16eb2-f983-4991-8d3f-a32e2e42a521-config-out\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.201011 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.195097 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-config-volume\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.201011 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.197344 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qb4j\" (UniqueName: \"kubernetes.io/projected/c3d16eb2-f983-4991-8d3f-a32e2e42a521-kube-api-access-4qb4j\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.201011 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.198093 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.202180 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.202080 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.203501 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.203423 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.684683 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.684638 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.688992 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.688923 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:46.861392 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:46.861255 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:08:47.132221 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:47.132136 2563 generic.go:356] "Generic (PLEG): container finished" podID="191a7f56-c678-4efa-8f90-8e1d6016085b" containerID="609f2557936158866bcf47f9c1890f4838039323f6e8cebfdd89f4add66501c4" exitCode=0 Sep 13 19:08:47.132221 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:47.132184 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lfttb" event={"ID":"191a7f56-c678-4efa-8f90-8e1d6016085b","Type":"ContainerDied","Data":"609f2557936158866bcf47f9c1890f4838039323f6e8cebfdd89f4add66501c4"} Sep 13 19:08:47.172775 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:47.172703 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 13 19:08:47.180800 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:47.180758 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d16eb2_f983_4991_8d3f_a32e2e42a521.slice/crio-1bc7e43a304d0c98241ee91d61310e71f8513d07538becb5bc1e9aeda2c2712c WatchSource:0}: Error finding container 1bc7e43a304d0c98241ee91d61310e71f8513d07538becb5bc1e9aeda2c2712c: Status 404 returned error can't find the container with id 1bc7e43a304d0c98241ee91d61310e71f8513d07538becb5bc1e9aeda2c2712c Sep 13 19:08:47.715211 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:47.715160 2563 patch_prober.go:28] interesting pod/image-registry-59698f7bd5-xn44h container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 13 19:08:47.715430 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:47.715230 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" podUID="7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 13 19:08:48.060835 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:48.060748 2563 patch_prober.go:28] interesting pod/image-registry-5569d8f464-5dzh9 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 13 19:08:48.061115 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:48.061069 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" podUID="1c10e26c-528d-4f49-806d-9c15d418e11c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 13 19:08:48.139618 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:48.139580 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lfttb" event={"ID":"191a7f56-c678-4efa-8f90-8e1d6016085b","Type":"ContainerStarted","Data":"ba579953d48a3e43f4297d8766ac79cbbe0b90112a80cacd9a3bdd6ed20c480b"} Sep 13 19:08:48.140095 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:48.139641 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lfttb" event={"ID":"191a7f56-c678-4efa-8f90-8e1d6016085b","Type":"ContainerStarted","Data":"0577c6578a2aace8614ec729707db1d8e9ced69bdabef7372c937e148f4376af"} Sep 13 19:08:48.141319 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:48.141286 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerStarted","Data":"1bc7e43a304d0c98241ee91d61310e71f8513d07538becb5bc1e9aeda2c2712c"} Sep 13 19:08:48.155766 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:48.155700 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lfttb" podStartSLOduration=2.698318972 podStartE2EDuration="4.155662638s" totalImagesPullingTime="1.34825052s" totalInitContainerRuntime="109.093146ms" isStatefulPod=true podCreationTimestamp="2026-09-13 19:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:48.154989525 +0000 UTC m=+279.536518556" watchObservedRunningTime="2026-09-13 19:08:48.155662638 +0000 UTC m=+279.537191667" Sep 13 19:08:49.151452 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.151387 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lkmtm" event={"ID":"64b72404-61ee-45b3-bfcd-fcf8a166664b","Type":"ContainerStarted","Data":"006dd4dcbeba9a00744d0325d5722b3bc6fa1472e7bcb72795bbd0b7dab893ab"} Sep 13 19:08:49.168253 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.168178 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lkmtm" podStartSLOduration=40.249811149 podStartE2EDuration="43.168161698s" totalImagesPullingTime="2.918350549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:08:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:49.166595505 +0000 UTC m=+280.548124535" watchObservedRunningTime="2026-09-13 19:08:49.168161698 +0000 UTC m=+280.549690733" Sep 13 19:08:49.241111 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.241070 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-c95ff8bbc-2vm78"] Sep 13 19:08:49.258904 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.258873 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c95ff8bbc-2vm78"] Sep 13 19:08:49.259086 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.258993 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.261614 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.261587 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 13 19:08:49.261770 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.261630 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 13 19:08:49.261931 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.261911 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-r8cmc\"" Sep 13 19:08:49.262103 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.262087 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 13 19:08:49.262374 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.262355 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-abf1bo3agbrrc\"" Sep 13 19:08:49.262456 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.262411 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 13 19:08:49.412739 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.412699 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-client-ca-bundle\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.412905 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.412802 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.412981 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.412900 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-secret-metrics-server-client-certs\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.412981 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.412954 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-audit-log\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.413083 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.413019 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjps9\" (UniqueName: \"kubernetes.io/projected/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-kube-api-access-cjps9\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.413083 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.413057 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-secret-metrics-server-tls\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.413177 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.413150 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-metrics-server-audit-profiles\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.514266 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.514201 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-metrics-server-audit-profiles\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.514455 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.514298 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-client-ca-bundle\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.514455 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.514344 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.514455 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.514395 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-secret-metrics-server-client-certs\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.514455 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.514432 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-audit-log\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.514663 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.514464 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjps9\" (UniqueName: \"kubernetes.io/projected/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-kube-api-access-cjps9\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.514663 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.514494 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-secret-metrics-server-tls\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.515148 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.515113 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-audit-log\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.515453 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.515400 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-metrics-server-audit-profiles\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.515575 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.515473 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.517837 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.517804 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-client-ca-bundle\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.517956 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.517907 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-secret-metrics-server-client-certs\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.518545 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.518509 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-secret-metrics-server-tls\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.523301 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.523262 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjps9\" (UniqueName: \"kubernetes.io/projected/cdbe965a-b0a3-4cfc-8312-b5e091f80ad9-kube-api-access-cjps9\") pod \"metrics-server-c95ff8bbc-2vm78\" (UID: \"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9\") " pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.572287 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.571859 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:08:49.737307 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:49.737251 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bvg5t" podUID="b17a3853-b777-4980-9a42-0e87a2c963f8" Sep 13 19:08:49.869980 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:49.869902 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c95ff8bbc-2vm78"] Sep 13 19:08:49.872847 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:49.872815 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdbe965a_b0a3_4cfc_8312_b5e091f80ad9.slice/crio-5f54a3744779419f0bb1104637174b3294f173d712aafc3c244e1ddb70cc20fb WatchSource:0}: Error finding container 5f54a3744779419f0bb1104637174b3294f173d712aafc3c244e1ddb70cc20fb: Status 404 returned error can't find the container with id 5f54a3744779419f0bb1104637174b3294f173d712aafc3c244e1ddb70cc20fb Sep 13 19:08:50.057594 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.057514 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5569d8f464-5dzh9" Sep 13 19:08:50.156385 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.156345 2563 generic.go:356] "Generic (PLEG): container finished" podID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerID="0fc579e7aa9786c2f5403fc7a72f045efcf9b9ca832b820c7ea1908f1c0fc606" exitCode=0 Sep 13 19:08:50.156939 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.156451 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerDied","Data":"0fc579e7aa9786c2f5403fc7a72f045efcf9b9ca832b820c7ea1908f1c0fc606"} Sep 13 19:08:50.158021 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.157994 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bvg5t" Sep 13 19:08:50.158021 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.158006 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" event={"ID":"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9","Type":"ContainerStarted","Data":"5f54a3744779419f0bb1104637174b3294f173d712aafc3c244e1ddb70cc20fb"} Sep 13 19:08:50.279896 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.279858 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c574b9b6f-kr8vp"] Sep 13 19:08:50.307354 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.307318 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c574b9b6f-kr8vp"] Sep 13 19:08:50.307589 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.307474 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.310454 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.310384 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 13 19:08:50.310454 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.310397 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 13 19:08:50.310454 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.310422 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 13 19:08:50.310677 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.310502 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 13 19:08:50.310950 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.310901 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hg8ww\"" Sep 13 19:08:50.311230 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.311206 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 13 19:08:50.315491 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.315463 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 13 19:08:50.423471 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.423435 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-config\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.423634 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.423488 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-serving-cert\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.423634 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.423521 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-service-ca\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.423634 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.423592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6cv6\" (UniqueName: \"kubernetes.io/projected/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-kube-api-access-t6cv6\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.423770 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.423640 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-oauth-serving-cert\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.423770 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.423719 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-trusted-ca-bundle\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.423770 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.423740 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-oauth-config\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.524893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.524848 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-oauth-serving-cert\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.525648 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.524938 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-trusted-ca-bundle\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.525648 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.524968 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-oauth-config\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.525648 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.525022 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-config\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.525648 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.525057 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-serving-cert\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.525648 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.525088 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-service-ca\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.525648 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.525139 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6cv6\" (UniqueName: \"kubernetes.io/projected/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-kube-api-access-t6cv6\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.526325 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.525758 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-oauth-serving-cert\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.526473 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.526451 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-service-ca\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.526996 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.526975 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-config\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.527210 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.527094 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-trusted-ca-bundle\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.529380 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.529190 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-serving-cert\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.529515 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.529487 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-oauth-config\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.534632 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.534600 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6cv6\" (UniqueName: \"kubernetes.io/projected/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-kube-api-access-t6cv6\") pod \"console-6c574b9b6f-kr8vp\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.621723 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.621349 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:08:50.946319 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:50.946219 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c574b9b6f-kr8vp"] Sep 13 19:08:50.950706 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:50.950669 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b3f9f4_a236_4ecc_a8d5_0ff6464624e2.slice/crio-0efd5e3293721530fb97d01bcb33e88d9af2d5cdcb9eaa019471030a16b1ae13 WatchSource:0}: Error finding container 0efd5e3293721530fb97d01bcb33e88d9af2d5cdcb9eaa019471030a16b1ae13: Status 404 returned error can't find the container with id 0efd5e3293721530fb97d01bcb33e88d9af2d5cdcb9eaa019471030a16b1ae13 Sep 13 19:08:51.163590 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:51.163542 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c574b9b6f-kr8vp" event={"ID":"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2","Type":"ContainerStarted","Data":"0efd5e3293721530fb97d01bcb33e88d9af2d5cdcb9eaa019471030a16b1ae13"} Sep 13 19:08:52.729450 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:52.729063 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" podUID="7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" containerName="registry" containerID="cri-o://fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e" gracePeriod=30 Sep 13 19:08:53.088819 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.088653 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:53.178434 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.178394 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerStarted","Data":"aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18"} Sep 13 19:08:53.178600 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.178484 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerStarted","Data":"6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629"} Sep 13 19:08:53.181525 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.180517 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" event={"ID":"cdbe965a-b0a3-4cfc-8312-b5e091f80ad9","Type":"ContainerStarted","Data":"7ff3abd11914edcec27434d5aca5e10b686ec8d048ba29e80b3a840456dacf73"} Sep 13 19:08:53.184569 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.183217 2563 generic.go:356] "Generic (PLEG): container finished" podID="7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" containerID="fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e" exitCode=0 Sep 13 19:08:53.184569 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.183299 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" event={"ID":"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48","Type":"ContainerDied","Data":"fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e"} Sep 13 19:08:53.184569 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.183338 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" event={"ID":"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48","Type":"ContainerDied","Data":"568e6b15529b4715964b80f3b5d9d15a0510d38cbd48cce1cd0763d597ff633a"} Sep 13 19:08:53.184569 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.183365 2563 scope.go:117] "RemoveContainer" containerID="fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e" Sep 13 19:08:53.184569 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.183523 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59698f7bd5-xn44h" Sep 13 19:08:53.197147 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.197036 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" podStartSLOduration=1.374563261 podStartE2EDuration="4.19701658s" totalImagesPullingTime="2.822453319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:53.196350172 +0000 UTC m=+284.577879201" watchObservedRunningTime="2026-09-13 19:08:53.19701658 +0000 UTC m=+284.578545626" Sep 13 19:08:53.209352 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.209314 2563 scope.go:117] "RemoveContainer" containerID="fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e" Sep 13 19:08:53.209850 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:08:53.209817 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e\": container with ID starting with fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e not found: ID does not exist" containerID="fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e" Sep 13 19:08:53.209956 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.209862 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e"} err="failed to get container status \"fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e\": rpc error: code = NotFound desc = could not find container \"fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e\": container with ID starting with fc0d320e68a2ca482ffb960460c7d5f11093ed3fba5cab1569c6b1ff9664529e not found: ID does not exist" Sep 13 19:08:53.255571 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.255535 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls\") pod \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " Sep 13 19:08:53.255730 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.255604 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-certificates\") pod \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " Sep 13 19:08:53.255730 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.255636 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-image-registry-private-configuration\") pod \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " Sep 13 19:08:53.255730 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.255668 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-ca-trust-extracted\") pod \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " Sep 13 19:08:53.255730 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.255691 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-trusted-ca\") pod \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " Sep 13 19:08:53.255730 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.255722 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-bound-sa-token\") pod \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " Sep 13 19:08:53.256031 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.255745 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-installation-pull-secrets\") pod \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " Sep 13 19:08:53.256031 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.255779 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szt7z\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-kube-api-access-szt7z\") pod \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\" (UID: \"7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48\") " Sep 13 19:08:53.257809 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.257594 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:08:53.257809 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.257777 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:08:53.263259 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.259552 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:08:53.263259 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.259711 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-kube-api-access-szt7z" (OuterVolumeSpecName: "kube-api-access-szt7z") pod "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48"). InnerVolumeSpecName "kube-api-access-szt7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:08:53.263678 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.263563 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:08:53.263678 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.263643 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:08:53.269420 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.266223 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:08:53.269420 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.268802 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" (UID: "7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 19:08:53.358287 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.357129 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:08:53.358287 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.357178 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-registry-certificates\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:08:53.358287 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.357194 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-image-registry-private-configuration\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:08:53.358287 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.357208 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-ca-trust-extracted\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:08:53.358287 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.357221 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-trusted-ca\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:08:53.358287 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.357258 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-bound-sa-token\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:08:53.358287 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.357272 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-installation-pull-secrets\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:08:53.358287 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.357287 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szt7z\" (UniqueName: \"kubernetes.io/projected/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48-kube-api-access-szt7z\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:08:53.458689 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.458099 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:08:53.464381 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.464309 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b17a3853-b777-4980-9a42-0e87a2c963f8-metrics-tls\") pod \"dns-default-bvg5t\" (UID: \"b17a3853-b777-4980-9a42-0e87a2c963f8\") " pod="openshift-dns/dns-default-bvg5t" Sep 13 19:08:53.517136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.517068 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59698f7bd5-xn44h"] Sep 13 19:08:53.521784 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.521748 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59698f7bd5-xn44h"] Sep 13 19:08:53.761713 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.761679 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kkpnt\"" Sep 13 19:08:53.769700 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:53.769664 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bvg5t" Sep 13 19:08:54.075389 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:54.075181 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bvg5t"] Sep 13 19:08:54.198766 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:54.198728 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerStarted","Data":"611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560"} Sep 13 19:08:54.198966 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:54.198851 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerStarted","Data":"cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c"} Sep 13 19:08:54.198966 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:54.198898 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerStarted","Data":"5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea"} Sep 13 19:08:55.063087 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:08:55.063053 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb17a3853_b777_4980_9a42_0e87a2c963f8.slice/crio-c7ac27a992a2a5d1a2d4b0719e38e0539c8a26c9208f2891610945f619528638 WatchSource:0}: Error finding container c7ac27a992a2a5d1a2d4b0719e38e0539c8a26c9208f2891610945f619528638: Status 404 returned error can't find the container with id c7ac27a992a2a5d1a2d4b0719e38e0539c8a26c9208f2891610945f619528638 Sep 13 19:08:55.132364 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:55.132329 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" path="/var/lib/kubelet/pods/7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48/volumes" Sep 13 19:08:55.204510 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:55.204469 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bvg5t" event={"ID":"b17a3853-b777-4980-9a42-0e87a2c963f8","Type":"ContainerStarted","Data":"c7ac27a992a2a5d1a2d4b0719e38e0539c8a26c9208f2891610945f619528638"} Sep 13 19:08:56.211045 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:56.211013 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c574b9b6f-kr8vp" event={"ID":"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2","Type":"ContainerStarted","Data":"d133688d1ca3944b3fee3caf06ae999f71c48b4d10439a89d8a9f4d15f744513"} Sep 13 19:08:56.227632 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:56.227574 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c574b9b6f-kr8vp" podStartSLOduration=1.7456092060000001 podStartE2EDuration="6.227556079s" totalImagesPullingTime="4.481946873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:08:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:56.226698525 +0000 UTC m=+287.608227555" watchObservedRunningTime="2026-09-13 19:08:56.227556079 +0000 UTC m=+287.609085107" Sep 13 19:08:57.218775 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:57.218732 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerStarted","Data":"95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9"} Sep 13 19:08:57.244868 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:57.244815 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.810863534 podStartE2EDuration="12.244795107s" totalImagesPullingTime="7.418743079s" totalInitContainerRuntime="15.188494ms" isStatefulPod=false podCreationTimestamp="2026-09-13 19:08:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:57.244458471 +0000 UTC m=+288.625987483" watchObservedRunningTime="2026-09-13 19:08:57.244795107 +0000 UTC m=+288.626324137" Sep 13 19:08:58.225016 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:58.224983 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bvg5t" event={"ID":"b17a3853-b777-4980-9a42-0e87a2c963f8","Type":"ContainerStarted","Data":"546cdee307287f1b0b12265a1912e84fcbd34f15e3795c1d62d4d265ed1d9f7e"} Sep 13 19:08:58.225481 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:58.225035 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bvg5t" event={"ID":"b17a3853-b777-4980-9a42-0e87a2c963f8","Type":"ContainerStarted","Data":"a01ed35b1cec763493bd1f696f1a734337d6d9908b7d6c818e75ae32f54ac922"} Sep 13 19:08:58.225481 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:58.225409 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bvg5t" Sep 13 19:08:58.240618 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:08:58.240566 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bvg5t" podStartSLOduration=253.070308078 podStartE2EDuration="4m15.240547212s" totalImagesPullingTime="2.170239134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:08:58.239413019 +0000 UTC m=+289.620942049" watchObservedRunningTime="2026-09-13 19:08:58.240547212 +0000 UTC m=+289.622076240" Sep 13 19:09:00.622316 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:00.622270 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:09:00.622764 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:00.622356 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:09:00.632726 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:00.631799 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:09:01.240479 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:01.240451 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:09:09.019300 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:09.019276 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/1.log" Sep 13 19:09:09.031277 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:09.031224 2563 kubelet.go:1648] "Image garbage collection succeeded" Sep 13 19:09:09.036937 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:09.036906 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/1.log" Sep 13 19:09:09.236326 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:09.236293 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bvg5t" Sep 13 19:09:09.572378 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:09.572335 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:09:09.572569 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:09.572405 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:09:22.320787 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:22.320752 2563 generic.go:356] "Generic (PLEG): container finished" podID="cc5e33f3-8e38-43dc-9f04-bef15c0d860c" containerID="3f5aa5ad02ac4c894a037d472f3b55512488754617311f79867f7540db35c864" exitCode=0 Sep 13 19:09:22.321199 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:22.320825 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" event={"ID":"cc5e33f3-8e38-43dc-9f04-bef15c0d860c","Type":"ContainerDied","Data":"3f5aa5ad02ac4c894a037d472f3b55512488754617311f79867f7540db35c864"} Sep 13 19:09:22.321199 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:22.321143 2563 scope.go:117] "RemoveContainer" containerID="3f5aa5ad02ac4c894a037d472f3b55512488754617311f79867f7540db35c864" Sep 13 19:09:23.326504 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:23.326468 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nsbnf" event={"ID":"cc5e33f3-8e38-43dc-9f04-bef15c0d860c","Type":"ContainerStarted","Data":"f6a8dd2cce376b4fbcf7276d0ab1974aa631dead545f9f75d5f5d0b9ae5385ea"} Sep 13 19:09:29.577849 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:29.577817 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:09:29.581826 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:09:29.581800 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-c95ff8bbc-2vm78" Sep 13 19:10:05.104057 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.103969 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 13 19:10:05.104593 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.104324 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="alertmanager" containerID="cri-o://6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629" gracePeriod=120 Sep 13 19:10:05.104593 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.104386 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="kube-rbac-proxy-metric" containerID="cri-o://611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560" gracePeriod=120 Sep 13 19:10:05.104593 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.104445 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="config-reloader" containerID="cri-o://aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18" gracePeriod=120 Sep 13 19:10:05.104593 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.104491 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="prom-label-proxy" containerID="cri-o://95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9" gracePeriod=120 Sep 13 19:10:05.104593 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.104417 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="kube-rbac-proxy-web" containerID="cri-o://5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea" gracePeriod=120 Sep 13 19:10:05.104852 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.104605 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="kube-rbac-proxy" containerID="cri-o://cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c" gracePeriod=120 Sep 13 19:10:05.500926 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.500891 2563 generic.go:356] "Generic (PLEG): container finished" podID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerID="95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9" exitCode=0 Sep 13 19:10:05.500926 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.500919 2563 generic.go:356] "Generic (PLEG): container finished" podID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerID="611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560" exitCode=0 Sep 13 19:10:05.500926 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.500925 2563 generic.go:356] "Generic (PLEG): container finished" podID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerID="cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c" exitCode=0 Sep 13 19:10:05.500926 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.500931 2563 generic.go:356] "Generic (PLEG): container finished" podID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerID="aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18" exitCode=0 Sep 13 19:10:05.500926 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.500936 2563 generic.go:356] "Generic (PLEG): container finished" podID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerID="6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629" exitCode=0 Sep 13 19:10:05.501221 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.500958 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerDied","Data":"95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9"} Sep 13 19:10:05.501221 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.501023 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerDied","Data":"611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560"} Sep 13 19:10:05.501221 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.501053 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerDied","Data":"cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c"} Sep 13 19:10:05.501221 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.501077 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerDied","Data":"aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18"} Sep 13 19:10:05.501221 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:05.501101 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerDied","Data":"6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629"} Sep 13 19:10:06.347690 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.347652 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.381802 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.381763 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-main-tls\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.381802 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.381797 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c3d16eb2-f983-4991-8d3f-a32e2e42a521-alertmanager-main-db\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.382041 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.381828 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3d16eb2-f983-4991-8d3f-a32e2e42a521-alertmanager-trusted-ca-bundle\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.382041 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.381859 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c3d16eb2-f983-4991-8d3f-a32e2e42a521-tls-assets\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.382041 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.381893 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.382041 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.381932 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy-web\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.382041 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.381956 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-cluster-tls-config\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.382041 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.381984 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.382041 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.382017 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c3d16eb2-f983-4991-8d3f-a32e2e42a521-config-out\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.382392 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.382055 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-web-config\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.382392 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.382085 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qb4j\" (UniqueName: \"kubernetes.io/projected/c3d16eb2-f983-4991-8d3f-a32e2e42a521-kube-api-access-4qb4j\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.382392 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.382151 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3d16eb2-f983-4991-8d3f-a32e2e42a521-metrics-client-ca\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.382392 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.382174 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-config-volume\") pod \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\" (UID: \"c3d16eb2-f983-4991-8d3f-a32e2e42a521\") " Sep 13 19:10:06.383036 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.382931 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d16eb2-f983-4991-8d3f-a32e2e42a521-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 19:10:06.383553 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.383406 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3d16eb2-f983-4991-8d3f-a32e2e42a521-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:10:06.383783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.383759 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3d16eb2-f983-4991-8d3f-a32e2e42a521-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:10:06.387398 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.385600 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-config-volume" (OuterVolumeSpecName: "config-volume") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:10:06.387398 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.385803 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:10:06.388794 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.388136 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3d16eb2-f983-4991-8d3f-a32e2e42a521-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:10:06.388794 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.388147 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:10:06.388794 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.388544 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:10:06.388794 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.388729 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3d16eb2-f983-4991-8d3f-a32e2e42a521-kube-api-access-4qb4j" (OuterVolumeSpecName: "kube-api-access-4qb4j") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "kube-api-access-4qb4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:10:06.389059 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.389026 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:10:06.389188 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.389109 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d16eb2-f983-4991-8d3f-a32e2e42a521-config-out" (OuterVolumeSpecName: "config-out") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 19:10:06.392981 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.392948 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:10:06.400190 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.400149 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-web-config" (OuterVolumeSpecName: "web-config") pod "c3d16eb2-f983-4991-8d3f-a32e2e42a521" (UID: "c3d16eb2-f983-4991-8d3f-a32e2e42a521"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:10:06.482818 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482780 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.482818 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482815 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c3d16eb2-f983-4991-8d3f-a32e2e42a521-config-out\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.482818 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482824 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-web-config\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.483042 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482833 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qb4j\" (UniqueName: \"kubernetes.io/projected/c3d16eb2-f983-4991-8d3f-a32e2e42a521-kube-api-access-4qb4j\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.483042 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482842 2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3d16eb2-f983-4991-8d3f-a32e2e42a521-metrics-client-ca\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.483042 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482850 2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-config-volume\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.483042 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482858 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-main-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.483042 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482868 2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c3d16eb2-f983-4991-8d3f-a32e2e42a521-alertmanager-main-db\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.483042 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482878 2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3d16eb2-f983-4991-8d3f-a32e2e42a521-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.483042 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482886 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c3d16eb2-f983-4991-8d3f-a32e2e42a521-tls-assets\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.483042 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482894 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.483042 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482903 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.483042 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.482910 2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c3d16eb2-f983-4991-8d3f-a32e2e42a521-cluster-tls-config\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:06.508175 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.508139 2563 generic.go:356] "Generic (PLEG): container finished" podID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerID="5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea" exitCode=0 Sep 13 19:10:06.508387 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.508223 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.508387 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.508225 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerDied","Data":"5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea"} Sep 13 19:10:06.508387 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.508338 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3d16eb2-f983-4991-8d3f-a32e2e42a521","Type":"ContainerDied","Data":"1bc7e43a304d0c98241ee91d61310e71f8513d07538becb5bc1e9aeda2c2712c"} Sep 13 19:10:06.508387 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.508386 2563 scope.go:117] "RemoveContainer" containerID="95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9" Sep 13 19:10:06.523094 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.523070 2563 scope.go:117] "RemoveContainer" containerID="611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560" Sep 13 19:10:06.530836 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.530812 2563 scope.go:117] "RemoveContainer" containerID="cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c" Sep 13 19:10:06.535380 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.535354 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 13 19:10:06.538488 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.538459 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 13 19:10:06.539700 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.539676 2563 scope.go:117] "RemoveContainer" containerID="5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea" Sep 13 19:10:06.547443 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.547417 2563 scope.go:117] "RemoveContainer" containerID="aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18" Sep 13 19:10:06.555831 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.555810 2563 scope.go:117] "RemoveContainer" containerID="6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629" Sep 13 19:10:06.562724 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.562693 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 13 19:10:06.563003 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.562989 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="kube-rbac-proxy" Sep 13 19:10:06.563063 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563007 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="kube-rbac-proxy" Sep 13 19:10:06.563063 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563017 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="kube-rbac-proxy-metric" Sep 13 19:10:06.563063 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563025 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="kube-rbac-proxy-metric" Sep 13 19:10:06.563063 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563038 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="alertmanager" Sep 13 19:10:06.563063 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563043 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="alertmanager" Sep 13 19:10:06.563063 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563049 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="prom-label-proxy" Sep 13 19:10:06.563063 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563054 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="prom-label-proxy" Sep 13 19:10:06.563063 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563061 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" containerName="registry" Sep 13 19:10:06.563063 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563067 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" containerName="registry" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563074 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="init-config-reloader" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563080 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="init-config-reloader" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563086 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="kube-rbac-proxy-web" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563091 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="kube-rbac-proxy-web" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563098 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="config-reloader" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563103 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="config-reloader" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563146 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="kube-rbac-proxy-web" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563151 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="kube-rbac-proxy" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563159 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eb4d4e7-a4ca-4bc6-a00a-c752f8c15c48" containerName="registry" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563165 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="prom-label-proxy" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563171 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="alertmanager" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563177 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="config-reloader" Sep 13 19:10:06.563367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.563181 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" containerName="kube-rbac-proxy-metric" Sep 13 19:10:06.564388 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.564364 2563 scope.go:117] "RemoveContainer" containerID="0fc579e7aa9786c2f5403fc7a72f045efcf9b9ca832b820c7ea1908f1c0fc606" Sep 13 19:10:06.566568 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.566547 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.569313 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.569116 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 13 19:10:06.569313 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.569125 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 13 19:10:06.569313 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.569179 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dsqcj\"" Sep 13 19:10:06.569313 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.569193 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 13 19:10:06.569313 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.569204 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 13 19:10:06.569313 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.569222 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 13 19:10:06.569642 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.569599 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 13 19:10:06.569642 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.569599 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 13 19:10:06.569735 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.569603 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 13 19:10:06.573883 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.573853 2563 scope.go:117] "RemoveContainer" containerID="95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9" Sep 13 19:10:06.574285 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.574263 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 13 19:10:06.574375 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:10:06.574304 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9\": container with ID starting with 95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9 not found: ID does not exist" containerID="95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9" Sep 13 19:10:06.574375 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.574333 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9"} err="failed to get container status \"95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9\": rpc error: code = NotFound desc = could not find container \"95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9\": container with ID starting with 95753d0f265ae60dea66d73b9ccfc062e8d7eb74116667a1ac30754773fb19c9 not found: ID does not exist" Sep 13 19:10:06.574375 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.574359 2563 scope.go:117] "RemoveContainer" containerID="611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560" Sep 13 19:10:06.574765 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:10:06.574739 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560\": container with ID starting with 611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560 not found: ID does not exist" containerID="611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560" Sep 13 19:10:06.574869 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.574772 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560"} err="failed to get container status \"611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560\": rpc error: code = NotFound desc = could not find container \"611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560\": container with ID starting with 611e7090476e1997b5df913fffe9ff7e3460cb0b828438664ff506c9d74c8560 not found: ID does not exist" Sep 13 19:10:06.574869 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.574793 2563 scope.go:117] "RemoveContainer" containerID="cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c" Sep 13 19:10:06.575083 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:10:06.575067 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c\": container with ID starting with cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c not found: ID does not exist" containerID="cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c" Sep 13 19:10:06.575143 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.575091 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c"} err="failed to get container status \"cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c\": rpc error: code = NotFound desc = could not find container \"cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c\": container with ID starting with cac22bb51dc3e4f49123cbff9cce28eab71be5749b866042f32d27e6973c5a4c not found: ID does not exist" Sep 13 19:10:06.575143 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.575110 2563 scope.go:117] "RemoveContainer" containerID="5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea" Sep 13 19:10:06.575481 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:10:06.575463 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea\": container with ID starting with 5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea not found: ID does not exist" containerID="5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea" Sep 13 19:10:06.575530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.575487 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea"} err="failed to get container status \"5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea\": rpc error: code = NotFound desc = could not find container \"5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea\": container with ID starting with 5d8efee8fadd6c8dee83383d718f85dec20bfb9896b44abc984e4213876201ea not found: ID does not exist" Sep 13 19:10:06.575530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.575503 2563 scope.go:117] "RemoveContainer" containerID="aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18" Sep 13 19:10:06.575750 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:10:06.575736 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18\": container with ID starting with aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18 not found: ID does not exist" containerID="aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18" Sep 13 19:10:06.575802 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.575751 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18"} err="failed to get container status \"aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18\": rpc error: code = NotFound desc = could not find container \"aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18\": container with ID starting with aa2b23cae5107644b3927a5b4b68bfd55c55e2568e7f36c41447c23a9a309f18 not found: ID does not exist" Sep 13 19:10:06.575802 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.575762 2563 scope.go:117] "RemoveContainer" containerID="6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629" Sep 13 19:10:06.576031 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:10:06.576012 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629\": container with ID starting with 6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629 not found: ID does not exist" containerID="6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629" Sep 13 19:10:06.576093 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.576041 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629"} err="failed to get container status \"6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629\": rpc error: code = NotFound desc = could not find container \"6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629\": container with ID starting with 6302491ffc3712dae8f87a4116fececc77063d401e293a35316d0b5a866b0629 not found: ID does not exist" Sep 13 19:10:06.576093 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.576061 2563 scope.go:117] "RemoveContainer" containerID="0fc579e7aa9786c2f5403fc7a72f045efcf9b9ca832b820c7ea1908f1c0fc606" Sep 13 19:10:06.576329 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:10:06.576313 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fc579e7aa9786c2f5403fc7a72f045efcf9b9ca832b820c7ea1908f1c0fc606\": container with ID starting with 0fc579e7aa9786c2f5403fc7a72f045efcf9b9ca832b820c7ea1908f1c0fc606 not found: ID does not exist" containerID="0fc579e7aa9786c2f5403fc7a72f045efcf9b9ca832b820c7ea1908f1c0fc606" Sep 13 19:10:06.576387 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.576333 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc579e7aa9786c2f5403fc7a72f045efcf9b9ca832b820c7ea1908f1c0fc606"} err="failed to get container status \"0fc579e7aa9786c2f5403fc7a72f045efcf9b9ca832b820c7ea1908f1c0fc606\": rpc error: code = NotFound desc = could not find container \"0fc579e7aa9786c2f5403fc7a72f045efcf9b9ca832b820c7ea1908f1c0fc606\": container with ID starting with 0fc579e7aa9786c2f5403fc7a72f045efcf9b9ca832b820c7ea1908f1c0fc606 not found: ID does not exist" Sep 13 19:10:06.579623 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.579601 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 13 19:10:06.583312 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583287 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-config-volume\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.583476 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583321 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.583476 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583348 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.583476 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.583476 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583451 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hhsv\" (UniqueName: \"kubernetes.io/projected/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-kube-api-access-7hhsv\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.583633 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583526 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.583633 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583581 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.583633 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583611 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.583745 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583663 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.583745 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583690 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-web-config\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.583745 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583716 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.583852 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583760 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-config-out\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.583852 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.583793 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.684569 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.684526 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.684751 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.684581 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.684751 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.684605 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.684751 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.684657 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hhsv\" (UniqueName: \"kubernetes.io/projected/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-kube-api-access-7hhsv\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.684751 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.684702 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.684931 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.684750 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.684931 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.684803 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.685024 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.684928 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.685024 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.684961 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-web-config\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.685024 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.684987 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.685178 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.685044 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-config-out\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.685178 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.685069 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.685178 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.685109 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-config-volume\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.685893 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.685865 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.687020 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.686984 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.687574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.687549 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.687884 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.687856 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.687976 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.687875 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.687976 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.687857 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.688097 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.688010 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-config-out\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.688153 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.688096 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.688205 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.688148 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-config-volume\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.688205 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.688152 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-web-config\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.689613 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.689591 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.689787 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.689770 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.695314 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.695270 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hhsv\" (UniqueName: \"kubernetes.io/projected/55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a-kube-api-access-7hhsv\") pod \"alertmanager-main-0\" (UID: \"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:06.878051 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:06.877967 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 13 19:10:07.129650 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:07.129565 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3d16eb2-f983-4991-8d3f-a32e2e42a521" path="/var/lib/kubelet/pods/c3d16eb2-f983-4991-8d3f-a32e2e42a521/volumes" Sep 13 19:10:07.156693 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:07.156660 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 13 19:10:07.160014 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:10:07.159991 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e57fc9_9a30_4b6a_a8a8_2c0efefc3c5a.slice/crio-da2d9581e1a488e8e26b1513805310115590a4f51ffa33fde7ff3b5490e0352b WatchSource:0}: Error finding container da2d9581e1a488e8e26b1513805310115590a4f51ffa33fde7ff3b5490e0352b: Status 404 returned error can't find the container with id da2d9581e1a488e8e26b1513805310115590a4f51ffa33fde7ff3b5490e0352b Sep 13 19:10:07.513542 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:07.513505 2563 generic.go:356] "Generic (PLEG): container finished" podID="55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a" containerID="5186ef190b97a837f174025fd6a3abbf2f56290d9cf5e5a0233f8124970123e4" exitCode=0 Sep 13 19:10:07.513963 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:07.513589 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a","Type":"ContainerDied","Data":"5186ef190b97a837f174025fd6a3abbf2f56290d9cf5e5a0233f8124970123e4"} Sep 13 19:10:07.513963 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:07.513666 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a","Type":"ContainerStarted","Data":"da2d9581e1a488e8e26b1513805310115590a4f51ffa33fde7ff3b5490e0352b"} Sep 13 19:10:08.522387 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:08.522353 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a","Type":"ContainerStarted","Data":"cc2ece9d279beac8c1dd873311cfa29238fe36f9dab75b258615f205dae6528f"} Sep 13 19:10:08.522783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:08.522421 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a","Type":"ContainerStarted","Data":"88665d8684765e0081ace605cca5c8b19de0283ebb2ec1580143163b9b3076a6"} Sep 13 19:10:08.522783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:08.522450 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a","Type":"ContainerStarted","Data":"b3f147a0d631f76adde3a5617953226e2918eab115690d47ec2ea4cf7e580e92"} Sep 13 19:10:08.522783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:08.522474 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a","Type":"ContainerStarted","Data":"d2f3ab30b88a7c965a4537cbabaa27675b856349154c8a6c8e383571bcfe3c7e"} Sep 13 19:10:08.522783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:08.522497 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a","Type":"ContainerStarted","Data":"98e2ae4de30bb6fab1729abe7d378dc0b872c44f082f4713fb6fbce301e0b253"} Sep 13 19:10:08.522783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:08.522519 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a","Type":"ContainerStarted","Data":"19e236b3d4e0f0ac74dc19bded08b307c652b5c2acf67456eb1d8f3ac75396c4"} Sep 13 19:10:08.547168 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:08.547121 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.529993807 podStartE2EDuration="2.547101833s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.108026ms" isStatefulPod=false podCreationTimestamp="2026-09-13 19:10:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:10:08.546700317 +0000 UTC m=+359.928229348" watchObservedRunningTime="2026-09-13 19:10:08.547101833 +0000 UTC m=+359.928630896" Sep 13 19:10:09.090233 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.090159 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-fffc965b6-pqs8x"] Sep 13 19:10:09.093932 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.093903 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.096539 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.096494 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 13 19:10:09.096708 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.096630 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zzc86\"" Sep 13 19:10:09.096834 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.096717 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 13 19:10:09.096834 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.096721 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 13 19:10:09.097019 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.096996 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 13 19:10:09.097065 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.097005 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 13 19:10:09.102295 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.102271 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 13 19:10:09.104628 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.104426 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5807642b-062c-4a3f-ae82-374a306acd53-telemeter-client-tls\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.104628 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.104503 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5807642b-062c-4a3f-ae82-374a306acd53-secret-telemeter-client\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.104628 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.104546 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5807642b-062c-4a3f-ae82-374a306acd53-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.104920 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.104663 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpvqj\" (UniqueName: \"kubernetes.io/projected/5807642b-062c-4a3f-ae82-374a306acd53-kube-api-access-vpvqj\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.104920 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.104690 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5807642b-062c-4a3f-ae82-374a306acd53-serving-certs-ca-bundle\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.104920 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.104712 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5807642b-062c-4a3f-ae82-374a306acd53-telemeter-trusted-ca-bundle\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.104920 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.104758 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5807642b-062c-4a3f-ae82-374a306acd53-metrics-client-ca\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.104920 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.104828 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5807642b-062c-4a3f-ae82-374a306acd53-federate-client-tls\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.105924 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.105901 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-fffc965b6-pqs8x"] Sep 13 19:10:09.205766 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.205725 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5807642b-062c-4a3f-ae82-374a306acd53-federate-client-tls\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.205766 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.205769 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5807642b-062c-4a3f-ae82-374a306acd53-telemeter-client-tls\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.205979 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.205898 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5807642b-062c-4a3f-ae82-374a306acd53-secret-telemeter-client\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.205979 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.205940 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5807642b-062c-4a3f-ae82-374a306acd53-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.206046 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.206002 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpvqj\" (UniqueName: \"kubernetes.io/projected/5807642b-062c-4a3f-ae82-374a306acd53-kube-api-access-vpvqj\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.206046 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.206020 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5807642b-062c-4a3f-ae82-374a306acd53-serving-certs-ca-bundle\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.206046 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.206035 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5807642b-062c-4a3f-ae82-374a306acd53-telemeter-trusted-ca-bundle\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.206192 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.206059 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5807642b-062c-4a3f-ae82-374a306acd53-metrics-client-ca\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.206877 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.206843 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5807642b-062c-4a3f-ae82-374a306acd53-serving-certs-ca-bundle\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.207008 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.206990 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5807642b-062c-4a3f-ae82-374a306acd53-telemeter-trusted-ca-bundle\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.207064 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.206988 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5807642b-062c-4a3f-ae82-374a306acd53-metrics-client-ca\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.208876 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.208852 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5807642b-062c-4a3f-ae82-374a306acd53-federate-client-tls\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.208984 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.208935 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5807642b-062c-4a3f-ae82-374a306acd53-telemeter-client-tls\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.209061 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.208987 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5807642b-062c-4a3f-ae82-374a306acd53-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.209061 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.208984 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5807642b-062c-4a3f-ae82-374a306acd53-secret-telemeter-client\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.213681 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.213649 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpvqj\" (UniqueName: \"kubernetes.io/projected/5807642b-062c-4a3f-ae82-374a306acd53-kube-api-access-vpvqj\") pod \"telemeter-client-fffc965b6-pqs8x\" (UID: \"5807642b-062c-4a3f-ae82-374a306acd53\") " pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.405985 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.405885 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" Sep 13 19:10:09.689020 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.688924 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-fffc965b6-pqs8x"] Sep 13 19:10:09.693434 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:10:09.693394 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5807642b_062c_4a3f_ae82_374a306acd53.slice/crio-173ee3f9ae7ad83854f360c8e06d1b1881a9cf9fe8c0d00099ab46246d25ac8e WatchSource:0}: Error finding container 173ee3f9ae7ad83854f360c8e06d1b1881a9cf9fe8c0d00099ab46246d25ac8e: Status 404 returned error can't find the container with id 173ee3f9ae7ad83854f360c8e06d1b1881a9cf9fe8c0d00099ab46246d25ac8e Sep 13 19:10:09.696405 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:09.696381 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 13 19:10:10.532563 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:10.532522 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" event={"ID":"5807642b-062c-4a3f-ae82-374a306acd53","Type":"ContainerStarted","Data":"173ee3f9ae7ad83854f360c8e06d1b1881a9cf9fe8c0d00099ab46246d25ac8e"} Sep 13 19:10:12.544407 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:12.544362 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" event={"ID":"5807642b-062c-4a3f-ae82-374a306acd53","Type":"ContainerStarted","Data":"64048c1f27a34e73ff4c5aa762e8d2de8fe9282c147deeacf662c595891a7542"} Sep 13 19:10:12.544810 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:12.544421 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" event={"ID":"5807642b-062c-4a3f-ae82-374a306acd53","Type":"ContainerStarted","Data":"c473baa8a88db13f3b6646e1464b431a1eb07725db91e6406cdc2f52d95ba4cb"} Sep 13 19:10:12.544810 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:12.544457 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" event={"ID":"5807642b-062c-4a3f-ae82-374a306acd53","Type":"ContainerStarted","Data":"1d34ccfd5d77794b70d84a23ddc680f171b552cb9bde50fd0512ae1811b16ece"} Sep 13 19:10:12.566749 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:12.566687 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-fffc965b6-pqs8x" podStartSLOduration=1.746452603 podStartE2EDuration="3.56667101s" totalImagesPullingTime="1.820218407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:10:12.565274945 +0000 UTC m=+363.946803974" watchObservedRunningTime="2026-09-13 19:10:12.56667101 +0000 UTC m=+363.948200039" Sep 13 19:10:23.454821 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:23.454765 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c574b9b6f-kr8vp"] Sep 13 19:10:48.475536 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.475478 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c574b9b6f-kr8vp" podUID="26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" containerName="console" containerID="cri-o://d133688d1ca3944b3fee3caf06ae999f71c48b4d10439a89d8a9f4d15f744513" gracePeriod=15 Sep 13 19:10:48.690691 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.690659 2563 generic.go:356] "Generic (PLEG): container finished" podID="26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" containerID="d133688d1ca3944b3fee3caf06ae999f71c48b4d10439a89d8a9f4d15f744513" exitCode=0 Sep 13 19:10:48.690903 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.690718 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c574b9b6f-kr8vp" event={"ID":"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2","Type":"ContainerDied","Data":"d133688d1ca3944b3fee3caf06ae999f71c48b4d10439a89d8a9f4d15f744513"} Sep 13 19:10:48.715437 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.715398 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:10:48.838346 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.838218 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-config\") pod \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " Sep 13 19:10:48.838346 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.838293 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-serving-cert\") pod \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " Sep 13 19:10:48.838346 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.838327 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-oauth-serving-cert\") pod \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " Sep 13 19:10:48.838621 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.838349 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-oauth-config\") pod \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " Sep 13 19:10:48.838621 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.838411 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6cv6\" (UniqueName: \"kubernetes.io/projected/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-kube-api-access-t6cv6\") pod \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " Sep 13 19:10:48.838621 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.838445 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-service-ca\") pod \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " Sep 13 19:10:48.838621 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.838475 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-trusted-ca-bundle\") pod \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\" (UID: \"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2\") " Sep 13 19:10:48.838824 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.838691 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-config" (OuterVolumeSpecName: "console-config") pod "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" (UID: "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:10:48.838920 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.838892 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" (UID: "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:10:48.839046 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.839022 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-service-ca" (OuterVolumeSpecName: "service-ca") pod "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" (UID: "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:10:48.839104 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.839043 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" (UID: "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:10:48.840814 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.840788 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-kube-api-access-t6cv6" (OuterVolumeSpecName: "kube-api-access-t6cv6") pod "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" (UID: "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2"). InnerVolumeSpecName "kube-api-access-t6cv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:10:48.840814 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.840795 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" (UID: "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:10:48.840814 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.840802 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" (UID: "26b3f9f4-a236-4ecc-a8d5-0ff6464624e2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:10:48.939902 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.939856 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6cv6\" (UniqueName: \"kubernetes.io/projected/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-kube-api-access-t6cv6\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:48.939902 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.939889 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-service-ca\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:48.939902 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.939900 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-trusted-ca-bundle\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:48.939902 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.939910 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-config\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:48.939902 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.939918 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-serving-cert\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:48.940201 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.939927 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-oauth-serving-cert\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:48.940201 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:48.939935 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2-console-oauth-config\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:10:49.695708 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:49.695673 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c574b9b6f-kr8vp" Sep 13 19:10:49.695708 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:49.695679 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c574b9b6f-kr8vp" event={"ID":"26b3f9f4-a236-4ecc-a8d5-0ff6464624e2","Type":"ContainerDied","Data":"0efd5e3293721530fb97d01bcb33e88d9af2d5cdcb9eaa019471030a16b1ae13"} Sep 13 19:10:49.696276 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:49.695762 2563 scope.go:117] "RemoveContainer" containerID="d133688d1ca3944b3fee3caf06ae999f71c48b4d10439a89d8a9f4d15f744513" Sep 13 19:10:49.712200 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:49.712163 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c574b9b6f-kr8vp"] Sep 13 19:10:49.715770 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:49.715736 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c574b9b6f-kr8vp"] Sep 13 19:10:51.129501 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:10:51.129457 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" path="/var/lib/kubelet/pods/26b3f9f4-a236-4ecc-a8d5-0ff6464624e2/volumes" Sep 13 19:11:27.970897 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.970853 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-545499f5c7-dghxj"] Sep 13 19:11:27.971454 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.971269 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" containerName="console" Sep 13 19:11:27.971454 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.971288 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" containerName="console" Sep 13 19:11:27.971454 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.971349 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="26b3f9f4-a236-4ecc-a8d5-0ff6464624e2" containerName="console" Sep 13 19:11:27.975630 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.975601 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:27.978432 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.978395 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 13 19:11:27.978579 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.978491 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 13 19:11:27.978579 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.978395 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 13 19:11:27.979336 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.979315 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hg8ww\"" Sep 13 19:11:27.979461 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.979315 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 13 19:11:27.979461 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.979384 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 13 19:11:27.983154 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.983125 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 13 19:11:27.988630 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:27.988601 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-545499f5c7-dghxj"] Sep 13 19:11:28.055229 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.055190 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ddf23723-2cb7-4564-92fe-a059399535d0-console-serving-cert\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.055432 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.055271 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-oauth-serving-cert\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.055432 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.055328 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ddf23723-2cb7-4564-92fe-a059399535d0-console-oauth-config\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.055432 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.055412 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-console-config\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.055543 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.055440 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-service-ca\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.055543 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.055470 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-trusted-ca-bundle\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.055543 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.055487 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkkh4\" (UniqueName: \"kubernetes.io/projected/ddf23723-2cb7-4564-92fe-a059399535d0-kube-api-access-gkkh4\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.156337 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.156285 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ddf23723-2cb7-4564-92fe-a059399535d0-console-oauth-config\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.156337 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.156347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-console-config\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.156608 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.156369 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-service-ca\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.156608 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.156406 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-trusted-ca-bundle\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.156608 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.156428 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkkh4\" (UniqueName: \"kubernetes.io/projected/ddf23723-2cb7-4564-92fe-a059399535d0-kube-api-access-gkkh4\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.156608 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.156484 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ddf23723-2cb7-4564-92fe-a059399535d0-console-serving-cert\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.156608 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.156519 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-oauth-serving-cert\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.157172 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.157130 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-console-config\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.157172 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.157148 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-service-ca\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.157403 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.157381 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-oauth-serving-cert\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.157749 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.157724 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-trusted-ca-bundle\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.158962 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.158941 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ddf23723-2cb7-4564-92fe-a059399535d0-console-oauth-config\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.159064 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.159049 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ddf23723-2cb7-4564-92fe-a059399535d0-console-serving-cert\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.163731 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.163709 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkkh4\" (UniqueName: \"kubernetes.io/projected/ddf23723-2cb7-4564-92fe-a059399535d0-kube-api-access-gkkh4\") pod \"console-545499f5c7-dghxj\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.288685 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.288593 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:28.561934 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.561913 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-545499f5c7-dghxj"] Sep 13 19:11:28.564342 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:11:28.564312 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf23723_2cb7_4564_92fe_a059399535d0.slice/crio-759bd370f681ed3f734b316d6a84c2d20de6e7b72ea80f9f5c0209e604136e19 WatchSource:0}: Error finding container 759bd370f681ed3f734b316d6a84c2d20de6e7b72ea80f9f5c0209e604136e19: Status 404 returned error can't find the container with id 759bd370f681ed3f734b316d6a84c2d20de6e7b72ea80f9f5c0209e604136e19 Sep 13 19:11:28.851674 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.851580 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-545499f5c7-dghxj" event={"ID":"ddf23723-2cb7-4564-92fe-a059399535d0","Type":"ContainerStarted","Data":"1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157"} Sep 13 19:11:28.851674 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.851631 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-545499f5c7-dghxj" event={"ID":"ddf23723-2cb7-4564-92fe-a059399535d0","Type":"ContainerStarted","Data":"759bd370f681ed3f734b316d6a84c2d20de6e7b72ea80f9f5c0209e604136e19"} Sep 13 19:11:28.867984 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:28.867932 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-545499f5c7-dghxj" podStartSLOduration=1.8679164259999999 podStartE2EDuration="1.867916426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:11:28.867030939 +0000 UTC m=+440.248559969" watchObservedRunningTime="2026-09-13 19:11:28.867916426 +0000 UTC m=+440.249445455" Sep 13 19:11:38.288976 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:38.288877 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:38.288976 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:38.288945 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:38.294230 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:38.294202 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:11:38.893801 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:11:38.893769 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:13:10.981655 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:10.981567 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9"] Sep 13 19:13:10.983865 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:10.983844 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9" Sep 13 19:13:10.986261 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:10.986215 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-72nnq\"" Sep 13 19:13:10.986405 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:10.986305 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 13 19:13:10.986405 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:10.986308 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 13 19:13:10.997792 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:10.997758 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9"] Sep 13 19:13:11.014123 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:11.014084 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7tvs\" (UniqueName: \"kubernetes.io/projected/4d88f528-3f5b-4d95-af19-50f85f432fe8-kube-api-access-v7tvs\") pod \"cert-manager-operator-controller-manager-55c444bf86-rf9g9\" (UID: \"4d88f528-3f5b-4d95-af19-50f85f432fe8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9" Sep 13 19:13:11.014315 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:11.014159 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d88f528-3f5b-4d95-af19-50f85f432fe8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rf9g9\" (UID: \"4d88f528-3f5b-4d95-af19-50f85f432fe8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9" Sep 13 19:13:11.115507 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:11.115461 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7tvs\" (UniqueName: \"kubernetes.io/projected/4d88f528-3f5b-4d95-af19-50f85f432fe8-kube-api-access-v7tvs\") pod \"cert-manager-operator-controller-manager-55c444bf86-rf9g9\" (UID: \"4d88f528-3f5b-4d95-af19-50f85f432fe8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9" Sep 13 19:13:11.115686 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:11.115558 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d88f528-3f5b-4d95-af19-50f85f432fe8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rf9g9\" (UID: \"4d88f528-3f5b-4d95-af19-50f85f432fe8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9" Sep 13 19:13:11.115982 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:11.115962 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d88f528-3f5b-4d95-af19-50f85f432fe8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rf9g9\" (UID: \"4d88f528-3f5b-4d95-af19-50f85f432fe8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9" Sep 13 19:13:11.123319 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:11.123292 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7tvs\" (UniqueName: \"kubernetes.io/projected/4d88f528-3f5b-4d95-af19-50f85f432fe8-kube-api-access-v7tvs\") pod \"cert-manager-operator-controller-manager-55c444bf86-rf9g9\" (UID: \"4d88f528-3f5b-4d95-af19-50f85f432fe8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9" Sep 13 19:13:11.292975 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:11.292888 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9" Sep 13 19:13:11.566597 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:11.566495 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9"] Sep 13 19:13:11.569587 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:13:11.569553 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d88f528_3f5b_4d95_af19_50f85f432fe8.slice/crio-e61b886fe50ace95c96075bd5044c5629a00a1c18145f3cfb273fb26d599e104 WatchSource:0}: Error finding container e61b886fe50ace95c96075bd5044c5629a00a1c18145f3cfb273fb26d599e104: Status 404 returned error can't find the container with id e61b886fe50ace95c96075bd5044c5629a00a1c18145f3cfb273fb26d599e104 Sep 13 19:13:12.260688 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:12.260643 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9" event={"ID":"4d88f528-3f5b-4d95-af19-50f85f432fe8","Type":"ContainerStarted","Data":"e61b886fe50ace95c96075bd5044c5629a00a1c18145f3cfb273fb26d599e104"} Sep 13 19:13:14.272031 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:14.271991 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9" event={"ID":"4d88f528-3f5b-4d95-af19-50f85f432fe8","Type":"ContainerStarted","Data":"39a395001e2daca1b91f9e023d430f79f64246a1e8634e5cbf2474ae3a1abfa6"} Sep 13 19:13:14.292197 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:14.292129 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rf9g9" podStartSLOduration=1.9417192509999999 podStartE2EDuration="4.292106475s" totalImagesPullingTime="2.350387224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:13:14.29006384 +0000 UTC m=+545.671592869" watchObservedRunningTime="2026-09-13 19:13:14.292106475 +0000 UTC m=+545.673635505" Sep 13 19:13:14.856149 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:14.856117 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zrfvx"] Sep 13 19:13:14.858306 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:14.858289 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" Sep 13 19:13:14.861109 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:14.861085 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 13 19:13:14.861262 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:14.861178 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 13 19:13:14.884101 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:14.884060 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zrfvx"] Sep 13 19:13:14.950055 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:14.950010 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zrfvx\" (UID: \"5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" Sep 13 19:13:14.950221 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:14.950124 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn6bg\" (UniqueName: \"kubernetes.io/projected/5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52-kube-api-access-jn6bg\") pod \"cert-manager-webhook-6486ffd997-zrfvx\" (UID: \"5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" Sep 13 19:13:15.051050 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:15.051014 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn6bg\" (UniqueName: \"kubernetes.io/projected/5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52-kube-api-access-jn6bg\") pod \"cert-manager-webhook-6486ffd997-zrfvx\" (UID: \"5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" Sep 13 19:13:15.051309 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:15.051065 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zrfvx\" (UID: \"5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" Sep 13 19:13:15.059647 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:15.059611 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zrfvx\" (UID: \"5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" Sep 13 19:13:15.059766 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:15.059680 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn6bg\" (UniqueName: \"kubernetes.io/projected/5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52-kube-api-access-jn6bg\") pod \"cert-manager-webhook-6486ffd997-zrfvx\" (UID: \"5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" Sep 13 19:13:15.181739 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:15.181639 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" Sep 13 19:13:15.455674 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:15.455644 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zrfvx"] Sep 13 19:13:15.458153 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:13:15.458128 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d1ea2aa_a7e4_4066_bfa4_8eaa6bb0ca52.slice/crio-5d7eed06aad819fae735b825adeddd80776677988a74ba6326ce98a8bda6c77f WatchSource:0}: Error finding container 5d7eed06aad819fae735b825adeddd80776677988a74ba6326ce98a8bda6c77f: Status 404 returned error can't find the container with id 5d7eed06aad819fae735b825adeddd80776677988a74ba6326ce98a8bda6c77f Sep 13 19:13:16.285728 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:16.285689 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" event={"ID":"5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52","Type":"ContainerStarted","Data":"5d7eed06aad819fae735b825adeddd80776677988a74ba6326ce98a8bda6c77f"} Sep 13 19:13:17.277934 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:17.277900 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk"] Sep 13 19:13:17.280220 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:17.280198 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk" Sep 13 19:13:17.282459 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:17.282432 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-76zl2\"" Sep 13 19:13:17.287881 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:17.287822 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk"] Sep 13 19:13:17.374897 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:17.374857 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f2d91d1-01ab-471b-84d7-47e5af3c479f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jjzfk\" (UID: \"0f2d91d1-01ab-471b-84d7-47e5af3c479f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk" Sep 13 19:13:17.375097 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:17.374939 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sp5m\" (UniqueName: \"kubernetes.io/projected/0f2d91d1-01ab-471b-84d7-47e5af3c479f-kube-api-access-6sp5m\") pod \"cert-manager-cainjector-559cdd8c87-jjzfk\" (UID: \"0f2d91d1-01ab-471b-84d7-47e5af3c479f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk" Sep 13 19:13:17.475517 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:17.475467 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sp5m\" (UniqueName: \"kubernetes.io/projected/0f2d91d1-01ab-471b-84d7-47e5af3c479f-kube-api-access-6sp5m\") pod \"cert-manager-cainjector-559cdd8c87-jjzfk\" (UID: \"0f2d91d1-01ab-471b-84d7-47e5af3c479f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk" Sep 13 19:13:17.475865 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:17.475835 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f2d91d1-01ab-471b-84d7-47e5af3c479f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jjzfk\" (UID: \"0f2d91d1-01ab-471b-84d7-47e5af3c479f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk" Sep 13 19:13:17.484661 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:17.484612 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f2d91d1-01ab-471b-84d7-47e5af3c479f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jjzfk\" (UID: \"0f2d91d1-01ab-471b-84d7-47e5af3c479f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk" Sep 13 19:13:17.484849 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:17.484749 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sp5m\" (UniqueName: \"kubernetes.io/projected/0f2d91d1-01ab-471b-84d7-47e5af3c479f-kube-api-access-6sp5m\") pod \"cert-manager-cainjector-559cdd8c87-jjzfk\" (UID: \"0f2d91d1-01ab-471b-84d7-47e5af3c479f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk" Sep 13 19:13:17.593628 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:17.593541 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk" Sep 13 19:13:18.144543 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:18.144371 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk"] Sep 13 19:13:18.147035 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:13:18.147011 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f2d91d1_01ab_471b_84d7_47e5af3c479f.slice/crio-487fb1c2aecedf905280bb22aeeb29591eb09822270f3187e5f6b4b88bd6599a WatchSource:0}: Error finding container 487fb1c2aecedf905280bb22aeeb29591eb09822270f3187e5f6b4b88bd6599a: Status 404 returned error can't find the container with id 487fb1c2aecedf905280bb22aeeb29591eb09822270f3187e5f6b4b88bd6599a Sep 13 19:13:18.298774 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:18.298733 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" event={"ID":"5d1ea2aa-a7e4-4066-bfa4-8eaa6bb0ca52","Type":"ContainerStarted","Data":"44a0c2f962c2ed7c4d6f0e6d0144f752aa44ce1eb54655136953fae9f4fc9203"} Sep 13 19:13:18.299197 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:18.298792 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" Sep 13 19:13:18.300767 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:18.300741 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk" event={"ID":"0f2d91d1-01ab-471b-84d7-47e5af3c479f","Type":"ContainerStarted","Data":"c08a819429e698ee8c5fa4a1b9df1bdf5439532c1aa68a40d93bfa0e7a7fd483"} Sep 13 19:13:18.300884 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:18.300782 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk" event={"ID":"0f2d91d1-01ab-471b-84d7-47e5af3c479f","Type":"ContainerStarted","Data":"487fb1c2aecedf905280bb22aeeb29591eb09822270f3187e5f6b4b88bd6599a"} Sep 13 19:13:18.315761 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:18.315706 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" podStartSLOduration=1.86306562 podStartE2EDuration="4.315686367s" totalImagesPullingTime="2.452620747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:13:18.314112295 +0000 UTC m=+549.695641347" watchObservedRunningTime="2026-09-13 19:13:18.315686367 +0000 UTC m=+549.697215395" Sep 13 19:13:18.330771 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:18.330711 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jjzfk" podStartSLOduration=1.330695754 podStartE2EDuration="1.330695754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:13:18.328592198 +0000 UTC m=+549.710121224" watchObservedRunningTime="2026-09-13 19:13:18.330695754 +0000 UTC m=+549.712224782" Sep 13 19:13:24.313079 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:24.313045 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zrfvx" Sep 13 19:13:51.391758 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.391715 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn"] Sep 13 19:13:51.396550 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.396525 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.399426 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.399394 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-z5lr9\"" Sep 13 19:13:51.399768 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.399749 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 13 19:13:51.400209 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.400188 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 13 19:13:51.400367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.400349 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 13 19:13:51.400604 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.400586 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 13 19:13:51.416936 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.416903 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn"] Sep 13 19:13:51.469799 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.469752 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61febc46-e159-491d-bf8d-8b7dc58175bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.469996 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.469815 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtlgh\" (UniqueName: \"kubernetes.io/projected/61febc46-e159-491d-bf8d-8b7dc58175bd-kube-api-access-rtlgh\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.469996 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.469870 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61febc46-e159-491d-bf8d-8b7dc58175bd-tmp\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.469996 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.469923 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61febc46-e159-491d-bf8d-8b7dc58175bd-manifests\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.469996 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.469956 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61febc46-e159-491d-bf8d-8b7dc58175bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.571162 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.571116 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61febc46-e159-491d-bf8d-8b7dc58175bd-manifests\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.571162 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.571165 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61febc46-e159-491d-bf8d-8b7dc58175bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.571409 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.571212 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61febc46-e159-491d-bf8d-8b7dc58175bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.571409 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.571284 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtlgh\" (UniqueName: \"kubernetes.io/projected/61febc46-e159-491d-bf8d-8b7dc58175bd-kube-api-access-rtlgh\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.571483 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.571408 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61febc46-e159-491d-bf8d-8b7dc58175bd-tmp\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.571682 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.571658 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61febc46-e159-491d-bf8d-8b7dc58175bd-manifests\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.573783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.573748 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61febc46-e159-491d-bf8d-8b7dc58175bd-tmp\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.574134 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.573988 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61febc46-e159-491d-bf8d-8b7dc58175bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.574134 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.574021 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61febc46-e159-491d-bf8d-8b7dc58175bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.579653 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.579610 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtlgh\" (UniqueName: \"kubernetes.io/projected/61febc46-e159-491d-bf8d-8b7dc58175bd-kube-api-access-rtlgh\") pod \"opendatahub-operator-controller-manager-65d87bcbcb-zm5vn\" (UID: \"61febc46-e159-491d-bf8d-8b7dc58175bd\") " pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.708559 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.708457 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:51.990457 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:51.990417 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn"] Sep 13 19:13:51.995126 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:13:51.995095 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61febc46_e159_491d_bf8d_8b7dc58175bd.slice/crio-7bd5751eda329b8d0d929f0bc7007415370f8884513e3a0b1ad51d86e4191e3e WatchSource:0}: Error finding container 7bd5751eda329b8d0d929f0bc7007415370f8884513e3a0b1ad51d86e4191e3e: Status 404 returned error can't find the container with id 7bd5751eda329b8d0d929f0bc7007415370f8884513e3a0b1ad51d86e4191e3e Sep 13 19:13:52.214452 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:52.214406 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-545499f5c7-dghxj"] Sep 13 19:13:52.457478 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:52.457383 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" event={"ID":"61febc46-e159-491d-bf8d-8b7dc58175bd","Type":"ContainerStarted","Data":"7bd5751eda329b8d0d929f0bc7007415370f8884513e3a0b1ad51d86e4191e3e"} Sep 13 19:13:55.475886 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:55.475842 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" event={"ID":"61febc46-e159-491d-bf8d-8b7dc58175bd","Type":"ContainerStarted","Data":"164132f7600ee276f30ca5e628fdc4f46aaed3ffeb34146d222c51a18b7fec63"} Sep 13 19:13:56.481728 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:56.481684 2563 generic.go:356] "Generic (PLEG): container finished" podID="61febc46-e159-491d-bf8d-8b7dc58175bd" containerID="164132f7600ee276f30ca5e628fdc4f46aaed3ffeb34146d222c51a18b7fec63" exitCode=0 Sep 13 19:13:56.482125 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:56.481737 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" event={"ID":"61febc46-e159-491d-bf8d-8b7dc58175bd","Type":"ContainerDied","Data":"164132f7600ee276f30ca5e628fdc4f46aaed3ffeb34146d222c51a18b7fec63"} Sep 13 19:13:57.488554 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:57.488518 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" event={"ID":"61febc46-e159-491d-bf8d-8b7dc58175bd","Type":"ContainerStarted","Data":"c9528f0f58a8c1af75fb44f43953dd57698fc1f5e71a532ea648730163a54edb"} Sep 13 19:13:57.488904 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:57.488661 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:13:57.511684 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:13:57.511625 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" podStartSLOduration=2.049403106 podStartE2EDuration="6.511605701s" totalImagesPullingTime="3.414050869s" totalInitContainerRuntime="1.048151726s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:13:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:13:57.511065926 +0000 UTC m=+588.892594954" watchObservedRunningTime="2026-09-13 19:13:57.511605701 +0000 UTC m=+588.893134731" Sep 13 19:14:08.496256 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:08.496198 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65d87bcbcb-zm5vn" Sep 13 19:14:09.051077 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:09.051050 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/1.log" Sep 13 19:14:09.067686 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:09.067658 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/1.log" Sep 13 19:14:15.623653 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.623612 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6"] Sep 13 19:14:15.626784 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.626765 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" Sep 13 19:14:15.629120 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.629088 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 13 19:14:15.629120 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.629111 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 13 19:14:15.629120 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.629096 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-l7hbr\"" Sep 13 19:14:15.630180 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.630163 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 13 19:14:15.630274 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.630191 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 13 19:14:15.636827 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.636802 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6"] Sep 13 19:14:15.684137 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.684101 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d18e47df-2426-4e8a-92c7-df3d0339fa1b-tls-certs\") pod \"kube-auth-proxy-56c745dfd9-blnv6\" (UID: \"d18e47df-2426-4e8a-92c7-df3d0339fa1b\") " pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" Sep 13 19:14:15.684371 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.684157 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d18e47df-2426-4e8a-92c7-df3d0339fa1b-tmp\") pod \"kube-auth-proxy-56c745dfd9-blnv6\" (UID: \"d18e47df-2426-4e8a-92c7-df3d0339fa1b\") " pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" Sep 13 19:14:15.684371 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.684231 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thmd4\" (UniqueName: \"kubernetes.io/projected/d18e47df-2426-4e8a-92c7-df3d0339fa1b-kube-api-access-thmd4\") pod \"kube-auth-proxy-56c745dfd9-blnv6\" (UID: \"d18e47df-2426-4e8a-92c7-df3d0339fa1b\") " pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" Sep 13 19:14:15.785144 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.785098 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d18e47df-2426-4e8a-92c7-df3d0339fa1b-tmp\") pod \"kube-auth-proxy-56c745dfd9-blnv6\" (UID: \"d18e47df-2426-4e8a-92c7-df3d0339fa1b\") " pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" Sep 13 19:14:15.785144 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.785156 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thmd4\" (UniqueName: \"kubernetes.io/projected/d18e47df-2426-4e8a-92c7-df3d0339fa1b-kube-api-access-thmd4\") pod \"kube-auth-proxy-56c745dfd9-blnv6\" (UID: \"d18e47df-2426-4e8a-92c7-df3d0339fa1b\") " pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" Sep 13 19:14:15.785449 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.785215 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d18e47df-2426-4e8a-92c7-df3d0339fa1b-tls-certs\") pod \"kube-auth-proxy-56c745dfd9-blnv6\" (UID: \"d18e47df-2426-4e8a-92c7-df3d0339fa1b\") " pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" Sep 13 19:14:15.787625 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.787594 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d18e47df-2426-4e8a-92c7-df3d0339fa1b-tmp\") pod \"kube-auth-proxy-56c745dfd9-blnv6\" (UID: \"d18e47df-2426-4e8a-92c7-df3d0339fa1b\") " pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" Sep 13 19:14:15.787829 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.787810 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d18e47df-2426-4e8a-92c7-df3d0339fa1b-tls-certs\") pod \"kube-auth-proxy-56c745dfd9-blnv6\" (UID: \"d18e47df-2426-4e8a-92c7-df3d0339fa1b\") " pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" Sep 13 19:14:15.799057 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.799025 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thmd4\" (UniqueName: \"kubernetes.io/projected/d18e47df-2426-4e8a-92c7-df3d0339fa1b-kube-api-access-thmd4\") pod \"kube-auth-proxy-56c745dfd9-blnv6\" (UID: \"d18e47df-2426-4e8a-92c7-df3d0339fa1b\") " pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" Sep 13 19:14:15.938155 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:15.938065 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" Sep 13 19:14:16.215821 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.215759 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6"] Sep 13 19:14:16.218629 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:14:16.218597 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd18e47df_2426_4e8a_92c7_df3d0339fa1b.slice/crio-13fe81f066fc1d36d7b51f784cd17775c6e189d26c5d3b764d0cd1d7880b9142 WatchSource:0}: Error finding container 13fe81f066fc1d36d7b51f784cd17775c6e189d26c5d3b764d0cd1d7880b9142: Status 404 returned error can't find the container with id 13fe81f066fc1d36d7b51f784cd17775c6e189d26c5d3b764d0cd1d7880b9142 Sep 13 19:14:16.577940 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.577853 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" event={"ID":"d18e47df-2426-4e8a-92c7-df3d0339fa1b","Type":"ContainerStarted","Data":"13fe81f066fc1d36d7b51f784cd17775c6e189d26c5d3b764d0cd1d7880b9142"} Sep 13 19:14:16.637417 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.637378 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g"] Sep 13 19:14:16.641959 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.641940 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.644225 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.644199 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 13 19:14:16.644363 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.644254 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 13 19:14:16.644593 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.644576 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jxwjh\"" Sep 13 19:14:16.644593 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.644581 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 13 19:14:16.644781 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.644642 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 13 19:14:16.653739 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.653711 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g"] Sep 13 19:14:16.692533 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.692493 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.692724 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.692547 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.692724 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.692602 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.692724 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.692624 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.692724 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.692651 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55pg5\" (UniqueName: \"kubernetes.io/projected/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-kube-api-access-55pg5\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.692724 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.692694 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.692905 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.692783 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.793847 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.793806 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.794032 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.793873 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.794032 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.793913 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.794032 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.793947 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.794032 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.793970 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.794032 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.793986 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55pg5\" (UniqueName: \"kubernetes.io/projected/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-kube-api-access-55pg5\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.794344 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.794109 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.794526 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.794492 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.796456 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.796426 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.796456 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.796437 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.796625 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.796473 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.796625 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.796512 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.801606 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.801579 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.801716 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.801624 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55pg5\" (UniqueName: \"kubernetes.io/projected/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-kube-api-access-55pg5\") pod \"istiod-openshift-gateway-7b7995d5cd-44m6g\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:16.951948 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:16.951860 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:17.235305 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.235261 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-545499f5c7-dghxj" podUID="ddf23723-2cb7-4564-92fe-a059399535d0" containerName="console" containerID="cri-o://1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157" gracePeriod=15 Sep 13 19:14:17.246042 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.245949 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g"] Sep 13 19:14:17.249832 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:14:17.249779 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b163b61_5ab2_4156_9f0e_6fe2d8bfc84a.slice/crio-af0a294f1d57cc17d44d2a675d7096659a2d6e35231215e45d12c61331dedcb0 WatchSource:0}: Error finding container af0a294f1d57cc17d44d2a675d7096659a2d6e35231215e45d12c61331dedcb0: Status 404 returned error can't find the container with id af0a294f1d57cc17d44d2a675d7096659a2d6e35231215e45d12c61331dedcb0 Sep 13 19:14:17.500936 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.500902 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:14:17.586730 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.586685 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" event={"ID":"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a","Type":"ContainerStarted","Data":"af0a294f1d57cc17d44d2a675d7096659a2d6e35231215e45d12c61331dedcb0"} Sep 13 19:14:17.588545 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.588514 2563 generic.go:356] "Generic (PLEG): container finished" podID="ddf23723-2cb7-4564-92fe-a059399535d0" containerID="1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157" exitCode=0 Sep 13 19:14:17.588702 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.588563 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-545499f5c7-dghxj" event={"ID":"ddf23723-2cb7-4564-92fe-a059399535d0","Type":"ContainerDied","Data":"1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157"} Sep 13 19:14:17.588702 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.588615 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-545499f5c7-dghxj" event={"ID":"ddf23723-2cb7-4564-92fe-a059399535d0","Type":"ContainerDied","Data":"759bd370f681ed3f734b316d6a84c2d20de6e7b72ea80f9f5c0209e604136e19"} Sep 13 19:14:17.588702 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.588643 2563 scope.go:117] "RemoveContainer" containerID="1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157" Sep 13 19:14:17.588863 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.588852 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-545499f5c7-dghxj" Sep 13 19:14:17.599542 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.599512 2563 scope.go:117] "RemoveContainer" containerID="1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157" Sep 13 19:14:17.599969 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:14:17.599936 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157\": container with ID starting with 1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157 not found: ID does not exist" containerID="1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157" Sep 13 19:14:17.600097 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.599979 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157"} err="failed to get container status \"1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157\": rpc error: code = NotFound desc = could not find container \"1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157\": container with ID starting with 1b943a38b6e40dfe084477d793127cc9e4ea0019d6c4706eb065bd4b19430157 not found: ID does not exist" Sep 13 19:14:17.604324 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.604297 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkkh4\" (UniqueName: \"kubernetes.io/projected/ddf23723-2cb7-4564-92fe-a059399535d0-kube-api-access-gkkh4\") pod \"ddf23723-2cb7-4564-92fe-a059399535d0\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " Sep 13 19:14:17.604498 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.604347 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-service-ca\") pod \"ddf23723-2cb7-4564-92fe-a059399535d0\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " Sep 13 19:14:17.604498 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.604376 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-oauth-serving-cert\") pod \"ddf23723-2cb7-4564-92fe-a059399535d0\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " Sep 13 19:14:17.604498 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.604408 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-console-config\") pod \"ddf23723-2cb7-4564-92fe-a059399535d0\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " Sep 13 19:14:17.604498 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.604432 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ddf23723-2cb7-4564-92fe-a059399535d0-console-oauth-config\") pod \"ddf23723-2cb7-4564-92fe-a059399535d0\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " Sep 13 19:14:17.604498 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.604472 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ddf23723-2cb7-4564-92fe-a059399535d0-console-serving-cert\") pod \"ddf23723-2cb7-4564-92fe-a059399535d0\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " Sep 13 19:14:17.604759 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.604505 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-trusted-ca-bundle\") pod \"ddf23723-2cb7-4564-92fe-a059399535d0\" (UID: \"ddf23723-2cb7-4564-92fe-a059399535d0\") " Sep 13 19:14:17.605190 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.605156 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ddf23723-2cb7-4564-92fe-a059399535d0" (UID: "ddf23723-2cb7-4564-92fe-a059399535d0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:14:17.605386 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.605265 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ddf23723-2cb7-4564-92fe-a059399535d0" (UID: "ddf23723-2cb7-4564-92fe-a059399535d0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:14:17.605452 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.605419 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-service-ca" (OuterVolumeSpecName: "service-ca") pod "ddf23723-2cb7-4564-92fe-a059399535d0" (UID: "ddf23723-2cb7-4564-92fe-a059399535d0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:14:17.605619 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.605566 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-console-config" (OuterVolumeSpecName: "console-config") pod "ddf23723-2cb7-4564-92fe-a059399535d0" (UID: "ddf23723-2cb7-4564-92fe-a059399535d0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:14:17.607132 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.607098 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf23723-2cb7-4564-92fe-a059399535d0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ddf23723-2cb7-4564-92fe-a059399535d0" (UID: "ddf23723-2cb7-4564-92fe-a059399535d0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:14:17.607514 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.607483 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddf23723-2cb7-4564-92fe-a059399535d0-kube-api-access-gkkh4" (OuterVolumeSpecName: "kube-api-access-gkkh4") pod "ddf23723-2cb7-4564-92fe-a059399535d0" (UID: "ddf23723-2cb7-4564-92fe-a059399535d0"). InnerVolumeSpecName "kube-api-access-gkkh4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:14:17.607598 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.607582 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf23723-2cb7-4564-92fe-a059399535d0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ddf23723-2cb7-4564-92fe-a059399535d0" (UID: "ddf23723-2cb7-4564-92fe-a059399535d0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:14:17.705619 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.705582 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkkh4\" (UniqueName: \"kubernetes.io/projected/ddf23723-2cb7-4564-92fe-a059399535d0-kube-api-access-gkkh4\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:17.705619 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.705614 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-service-ca\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:17.705619 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.705624 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-oauth-serving-cert\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:17.706250 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.705633 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-console-config\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:17.706250 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.705642 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ddf23723-2cb7-4564-92fe-a059399535d0-console-oauth-config\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:17.706250 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.705649 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ddf23723-2cb7-4564-92fe-a059399535d0-console-serving-cert\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:17.706250 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.705657 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf23723-2cb7-4564-92fe-a059399535d0-trusted-ca-bundle\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:17.915175 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.915114 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-545499f5c7-dghxj"] Sep 13 19:14:17.917120 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:17.917090 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-545499f5c7-dghxj"] Sep 13 19:14:19.132136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:19.132094 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddf23723-2cb7-4564-92fe-a059399535d0" path="/var/lib/kubelet/pods/ddf23723-2cb7-4564-92fe-a059399535d0/volumes" Sep 13 19:14:19.602521 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:19.602463 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" event={"ID":"d18e47df-2426-4e8a-92c7-df3d0339fa1b","Type":"ContainerStarted","Data":"447307edfe5d73b0f653ea8bf7514ac41ead800ce4e13c939664a17d835fb47c"} Sep 13 19:14:19.618029 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:19.617962 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-56c745dfd9-blnv6" podStartSLOduration=1.796576664 podStartE2EDuration="4.617937646s" totalImagesPullingTime="2.821360982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:14:19.615817502 +0000 UTC m=+610.997346532" watchObservedRunningTime="2026-09-13 19:14:19.617937646 +0000 UTC m=+610.999466674" Sep 13 19:14:20.870830 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:20.870792 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 13 19:14:20.871099 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:20.870859 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 13 19:14:21.114025 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.113983 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6gffv"] Sep 13 19:14:21.114349 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.114336 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddf23723-2cb7-4564-92fe-a059399535d0" containerName="console" Sep 13 19:14:21.114417 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.114351 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddf23723-2cb7-4564-92fe-a059399535d0" containerName="console" Sep 13 19:14:21.114450 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.114420 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddf23723-2cb7-4564-92fe-a059399535d0" containerName="console" Sep 13 19:14:21.117529 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.117503 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" Sep 13 19:14:21.119899 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.119821 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 13 19:14:21.119899 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.119854 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-bt6jk\"" Sep 13 19:14:21.132919 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.132885 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6gffv"] Sep 13 19:14:21.240768 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.240730 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k65x\" (UniqueName: \"kubernetes.io/projected/582cefd2-87ac-4225-b77e-26df9795f169-kube-api-access-7k65x\") pod \"odh-model-controller-7998c67dc4-6gffv\" (UID: \"582cefd2-87ac-4225-b77e-26df9795f169\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" Sep 13 19:14:21.240948 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.240916 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/582cefd2-87ac-4225-b77e-26df9795f169-cert\") pod \"odh-model-controller-7998c67dc4-6gffv\" (UID: \"582cefd2-87ac-4225-b77e-26df9795f169\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" Sep 13 19:14:21.341796 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.341757 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/582cefd2-87ac-4225-b77e-26df9795f169-cert\") pod \"odh-model-controller-7998c67dc4-6gffv\" (UID: \"582cefd2-87ac-4225-b77e-26df9795f169\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" Sep 13 19:14:21.341976 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.341824 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k65x\" (UniqueName: \"kubernetes.io/projected/582cefd2-87ac-4225-b77e-26df9795f169-kube-api-access-7k65x\") pod \"odh-model-controller-7998c67dc4-6gffv\" (UID: \"582cefd2-87ac-4225-b77e-26df9795f169\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" Sep 13 19:14:21.344484 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.344459 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/582cefd2-87ac-4225-b77e-26df9795f169-cert\") pod \"odh-model-controller-7998c67dc4-6gffv\" (UID: \"582cefd2-87ac-4225-b77e-26df9795f169\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" Sep 13 19:14:21.349613 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.349582 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k65x\" (UniqueName: \"kubernetes.io/projected/582cefd2-87ac-4225-b77e-26df9795f169-kube-api-access-7k65x\") pod \"odh-model-controller-7998c67dc4-6gffv\" (UID: \"582cefd2-87ac-4225-b77e-26df9795f169\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" Sep 13 19:14:21.429018 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.428924 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" Sep 13 19:14:21.615762 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.615720 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" event={"ID":"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a","Type":"ContainerStarted","Data":"c3f1f6dab95d4ea9c45796f3c8ec4e429512a0b3438f776e8ef92ea8e4f3fe4e"} Sep 13 19:14:21.615959 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.615900 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:21.617677 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.617647 2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-44m6g container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 13 19:14:21.617834 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.617716 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" podUID="3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 13 19:14:21.634212 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.634151 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" podStartSLOduration=2.104603599 podStartE2EDuration="5.634132635s" totalImagesPullingTime="3.529529036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:14:21.632321893 +0000 UTC m=+613.013850923" watchObservedRunningTime="2026-09-13 19:14:21.634132635 +0000 UTC m=+613.015661664" Sep 13 19:14:21.702757 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:21.702662 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6gffv"] Sep 13 19:14:21.707407 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:14:21.707371 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod582cefd2_87ac_4225_b77e_26df9795f169.slice/crio-96384ef27e61e784d11259afc2f99ee8ec07deae290876235914cf0ceb43420d WatchSource:0}: Error finding container 96384ef27e61e784d11259afc2f99ee8ec07deae290876235914cf0ceb43420d: Status 404 returned error can't find the container with id 96384ef27e61e784d11259afc2f99ee8ec07deae290876235914cf0ceb43420d Sep 13 19:14:22.263450 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.263414 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r"] Sep 13 19:14:22.268138 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.268112 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.270506 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.270479 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-c2ch5\"" Sep 13 19:14:22.279808 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.279781 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r"] Sep 13 19:14:22.352153 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.352108 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.352371 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.352191 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.352371 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.352233 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.352371 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.352312 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.352371 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.352343 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.352551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.352417 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtvfw\" (UniqueName: \"kubernetes.io/projected/f45303d0-58d5-4a0d-890f-c9893e7d28d7-kube-api-access-jtvfw\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.352551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.352458 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.352551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.352481 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.352551 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.352507 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.454468 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.454420 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.454649 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.454514 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.454649 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.454561 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.454649 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.454594 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.454649 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.454645 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtvfw\" (UniqueName: \"kubernetes.io/projected/f45303d0-58d5-4a0d-890f-c9893e7d28d7-kube-api-access-jtvfw\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.454846 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.454678 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.454846 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.454701 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.454846 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.454724 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.454846 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.454775 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.455041 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.454911 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.455166 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.455139 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.455340 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.455191 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.455340 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.455209 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.455735 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.455711 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.458367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.458311 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.458632 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.458607 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.469300 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.469273 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtvfw\" (UniqueName: \"kubernetes.io/projected/f45303d0-58d5-4a0d-890f-c9893e7d28d7-kube-api-access-jtvfw\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.470151 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.470123 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45303d0-58d5-4a0d-890f-c9893e7d28d7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r\" (UID: \"f45303d0-58d5-4a0d-890f-c9893e7d28d7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.579890 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.579796 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:22.623046 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.623002 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" event={"ID":"582cefd2-87ac-4225-b77e-26df9795f169","Type":"ContainerStarted","Data":"96384ef27e61e784d11259afc2f99ee8ec07deae290876235914cf0ceb43420d"} Sep 13 19:14:22.623989 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.623965 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:22.867977 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:22.867951 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r"] Sep 13 19:14:22.870565 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:14:22.870528 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf45303d0_58d5_4a0d_890f_c9893e7d28d7.slice/crio-912e8b3a3a09bdcc0615fc38129e95fcf1a6d321b22160ce95048ca1ae6b1d96 WatchSource:0}: Error finding container 912e8b3a3a09bdcc0615fc38129e95fcf1a6d321b22160ce95048ca1ae6b1d96: Status 404 returned error can't find the container with id 912e8b3a3a09bdcc0615fc38129e95fcf1a6d321b22160ce95048ca1ae6b1d96 Sep 13 19:14:23.629706 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:23.629667 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" event={"ID":"f45303d0-58d5-4a0d-890f-c9893e7d28d7","Type":"ContainerStarted","Data":"912e8b3a3a09bdcc0615fc38129e95fcf1a6d321b22160ce95048ca1ae6b1d96"} Sep 13 19:14:25.644430 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:25.644387 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" event={"ID":"582cefd2-87ac-4225-b77e-26df9795f169","Type":"ContainerStarted","Data":"7dca3fe4827c8183f1853ddbfdb83c800070cdaa638db5368c10738ddfbc4ac5"} Sep 13 19:14:25.644921 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:25.644572 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" Sep 13 19:14:25.659556 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:25.659494 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" podStartSLOduration=1.782010678 podStartE2EDuration="4.659475732s" totalImagesPullingTime="2.877465054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:14:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:14:25.65853358 +0000 UTC m=+617.040062610" watchObservedRunningTime="2026-09-13 19:14:25.659475732 +0000 UTC m=+617.041004762" Sep 13 19:14:26.526855 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:26.526823 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 13 19:14:26.526947 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:26.526900 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 13 19:14:26.526947 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:26.526932 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 13 19:14:26.663461 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:26.663419 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" event={"ID":"f45303d0-58d5-4a0d-890f-c9893e7d28d7","Type":"ContainerStarted","Data":"f7ccda368eff3a0250dac8d8274e96e2476105f1e9729e4e5fbfd392cf16a223"} Sep 13 19:14:26.682611 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:26.682542 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" podStartSLOduration=1.02928933 podStartE2EDuration="4.682522012s" totalImagesPullingTime="3.653232682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:14:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:14:26.680766875 +0000 UTC m=+618.062295910" watchObservedRunningTime="2026-09-13 19:14:26.682522012 +0000 UTC m=+618.064051186" Sep 13 19:14:27.580165 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:27.580122 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:27.585131 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:27.585103 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:27.668479 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:27.668438 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:27.669446 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:27.669424 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r" Sep 13 19:14:30.763331 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.763295 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-98n7v"] Sep 13 19:14:30.771962 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.771929 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" Sep 13 19:14:30.774443 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.774402 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4"] Sep 13 19:14:30.775665 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.775640 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 13 19:14:30.776165 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.776080 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mnh6q\"" Sep 13 19:14:30.778709 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.778669 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-98n7v"] Sep 13 19:14:30.778903 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.778804 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" Sep 13 19:14:30.781563 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.781485 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 13 19:14:30.782012 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.781668 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dsbn4\"" Sep 13 19:14:30.787803 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.787770 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4"] Sep 13 19:14:30.943979 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.943938 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8ccs\" (UniqueName: \"kubernetes.io/projected/094e442e-e861-4888-b314-9c6233b4b321-kube-api-access-s8ccs\") pod \"kserve-controller-manager-d6f4b65b8-98n7v\" (UID: \"094e442e-e861-4888-b314-9c6233b4b321\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" Sep 13 19:14:30.944152 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.943989 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7psl2\" (UniqueName: \"kubernetes.io/projected/698395ff-2ac6-4085-98b7-d702909cfff6-kube-api-access-7psl2\") pod \"llmisvc-controller-manager-777c6bd7d5-f4xt4\" (UID: \"698395ff-2ac6-4085-98b7-d702909cfff6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" Sep 13 19:14:30.944152 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.944022 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/094e442e-e861-4888-b314-9c6233b4b321-cert\") pod \"kserve-controller-manager-d6f4b65b8-98n7v\" (UID: \"094e442e-e861-4888-b314-9c6233b4b321\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" Sep 13 19:14:30.944152 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:30.944045 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/698395ff-2ac6-4085-98b7-d702909cfff6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f4xt4\" (UID: \"698395ff-2ac6-4085-98b7-d702909cfff6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" Sep 13 19:14:31.045372 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.045212 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8ccs\" (UniqueName: \"kubernetes.io/projected/094e442e-e861-4888-b314-9c6233b4b321-kube-api-access-s8ccs\") pod \"kserve-controller-manager-d6f4b65b8-98n7v\" (UID: \"094e442e-e861-4888-b314-9c6233b4b321\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" Sep 13 19:14:31.045372 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.045306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7psl2\" (UniqueName: \"kubernetes.io/projected/698395ff-2ac6-4085-98b7-d702909cfff6-kube-api-access-7psl2\") pod \"llmisvc-controller-manager-777c6bd7d5-f4xt4\" (UID: \"698395ff-2ac6-4085-98b7-d702909cfff6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" Sep 13 19:14:31.045372 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.045332 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/094e442e-e861-4888-b314-9c6233b4b321-cert\") pod \"kserve-controller-manager-d6f4b65b8-98n7v\" (UID: \"094e442e-e861-4888-b314-9c6233b4b321\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" Sep 13 19:14:31.045600 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:14:31.045480 2563 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Sep 13 19:14:31.045600 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.045502 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/698395ff-2ac6-4085-98b7-d702909cfff6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f4xt4\" (UID: \"698395ff-2ac6-4085-98b7-d702909cfff6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" Sep 13 19:14:31.045600 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:14:31.045537 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/094e442e-e861-4888-b314-9c6233b4b321-cert podName:094e442e-e861-4888-b314-9c6233b4b321 nodeName:}" failed. No retries permitted until 2026-09-13 19:14:31.545521654 +0000 UTC m=+622.927050696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/094e442e-e861-4888-b314-9c6233b4b321-cert") pod "kserve-controller-manager-d6f4b65b8-98n7v" (UID: "094e442e-e861-4888-b314-9c6233b4b321") : secret "kserve-webhook-server-cert" not found Sep 13 19:14:31.048397 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.048359 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/698395ff-2ac6-4085-98b7-d702909cfff6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f4xt4\" (UID: \"698395ff-2ac6-4085-98b7-d702909cfff6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" Sep 13 19:14:31.054040 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.054011 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8ccs\" (UniqueName: \"kubernetes.io/projected/094e442e-e861-4888-b314-9c6233b4b321-kube-api-access-s8ccs\") pod \"kserve-controller-manager-d6f4b65b8-98n7v\" (UID: \"094e442e-e861-4888-b314-9c6233b4b321\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" Sep 13 19:14:31.054433 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.054406 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7psl2\" (UniqueName: \"kubernetes.io/projected/698395ff-2ac6-4085-98b7-d702909cfff6-kube-api-access-7psl2\") pod \"llmisvc-controller-manager-777c6bd7d5-f4xt4\" (UID: \"698395ff-2ac6-4085-98b7-d702909cfff6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" Sep 13 19:14:31.106137 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.106089 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" Sep 13 19:14:31.399045 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.399015 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4"] Sep 13 19:14:31.401655 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:14:31.401621 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod698395ff_2ac6_4085_98b7_d702909cfff6.slice/crio-c0a0ec26d791fb6d9276ee60c19df767d79ecc291a593f6e846619455040741b WatchSource:0}: Error finding container c0a0ec26d791fb6d9276ee60c19df767d79ecc291a593f6e846619455040741b: Status 404 returned error can't find the container with id c0a0ec26d791fb6d9276ee60c19df767d79ecc291a593f6e846619455040741b Sep 13 19:14:31.552093 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.552056 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/094e442e-e861-4888-b314-9c6233b4b321-cert\") pod \"kserve-controller-manager-d6f4b65b8-98n7v\" (UID: \"094e442e-e861-4888-b314-9c6233b4b321\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" Sep 13 19:14:31.554601 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.554575 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/094e442e-e861-4888-b314-9c6233b4b321-cert\") pod \"kserve-controller-manager-d6f4b65b8-98n7v\" (UID: \"094e442e-e861-4888-b314-9c6233b4b321\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" Sep 13 19:14:31.689563 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.689470 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" event={"ID":"698395ff-2ac6-4085-98b7-d702909cfff6","Type":"ContainerStarted","Data":"c0a0ec26d791fb6d9276ee60c19df767d79ecc291a593f6e846619455040741b"} Sep 13 19:14:31.691655 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.691631 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" Sep 13 19:14:31.965923 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:31.965897 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-98n7v"] Sep 13 19:14:32.002229 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:14:32.002182 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod094e442e_e861_4888_b314_9c6233b4b321.slice/crio-43e56ca9f45a9acdfe4d16c596a9f4ebb6363881255573e99865c9d5b5b7be49 WatchSource:0}: Error finding container 43e56ca9f45a9acdfe4d16c596a9f4ebb6363881255573e99865c9d5b5b7be49: Status 404 returned error can't find the container with id 43e56ca9f45a9acdfe4d16c596a9f4ebb6363881255573e99865c9d5b5b7be49 Sep 13 19:14:32.697370 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:32.697323 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" event={"ID":"094e442e-e861-4888-b314-9c6233b4b321","Type":"ContainerStarted","Data":"43e56ca9f45a9acdfe4d16c596a9f4ebb6363881255573e99865c9d5b5b7be49"} Sep 13 19:14:33.430269 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.430207 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g"] Sep 13 19:14:33.430780 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.430478 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" podUID="3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" containerName="discovery" containerID="cri-o://c3f1f6dab95d4ea9c45796f3c8ec4e429512a0b3438f776e8ef92ea8e4f3fe4e" gracePeriod=30 Sep 13 19:14:33.705178 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.705140 2563 generic.go:356] "Generic (PLEG): container finished" podID="3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" containerID="c3f1f6dab95d4ea9c45796f3c8ec4e429512a0b3438f776e8ef92ea8e4f3fe4e" exitCode=0 Sep 13 19:14:33.705381 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.705206 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" event={"ID":"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a","Type":"ContainerDied","Data":"c3f1f6dab95d4ea9c45796f3c8ec4e429512a0b3438f776e8ef92ea8e4f3fe4e"} Sep 13 19:14:33.705381 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.705289 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" event={"ID":"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a","Type":"ContainerDied","Data":"af0a294f1d57cc17d44d2a675d7096659a2d6e35231215e45d12c61331dedcb0"} Sep 13 19:14:33.705381 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.705313 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af0a294f1d57cc17d44d2a675d7096659a2d6e35231215e45d12c61331dedcb0" Sep 13 19:14:33.705638 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.705620 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:33.876708 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.876665 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-csr-dns-cert\") pod \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " Sep 13 19:14:33.876894 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.876716 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-cacerts\") pod \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " Sep 13 19:14:33.876894 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.876755 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-kubeconfig\") pod \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " Sep 13 19:14:33.876894 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.876777 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-csr-ca-configmap\") pod \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " Sep 13 19:14:33.876894 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.876885 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-token\") pod \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " Sep 13 19:14:33.877119 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.876932 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55pg5\" (UniqueName: \"kubernetes.io/projected/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-kube-api-access-55pg5\") pod \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " Sep 13 19:14:33.877119 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.876985 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-local-certs\") pod \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\" (UID: \"3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a\") " Sep 13 19:14:33.877289 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.877258 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" (UID: "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:14:33.879929 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.879884 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-cacerts" (OuterVolumeSpecName: "cacerts") pod "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" (UID: "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:14:33.880058 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.880016 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-kube-api-access-55pg5" (OuterVolumeSpecName: "kube-api-access-55pg5") pod "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" (UID: "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a"). InnerVolumeSpecName "kube-api-access-55pg5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:14:33.880136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.880099 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" (UID: "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:14:33.880189 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.880174 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" (UID: "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:14:33.880260 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.880226 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-local-certs" (OuterVolumeSpecName: "local-certs") pod "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" (UID: "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 19:14:33.880318 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.880290 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" (UID: "3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:14:33.978018 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.977975 2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-csr-dns-cert\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:33.978018 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.978016 2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-cacerts\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:33.978293 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.978032 2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-kubeconfig\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:33.978293 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.978049 2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-csr-ca-configmap\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:33.978293 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.978084 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-istio-token\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:33.978293 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.978101 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55pg5\" (UniqueName: \"kubernetes.io/projected/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-kube-api-access-55pg5\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:33.978293 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:33.978115 2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a-local-certs\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:14:34.710612 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:34.710582 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g" Sep 13 19:14:34.735632 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:34.735588 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g"] Sep 13 19:14:34.739361 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:34.739327 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-44m6g"] Sep 13 19:14:35.132166 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:35.132126 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" path="/var/lib/kubelet/pods/3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a/volumes" Sep 13 19:14:35.717553 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:35.717507 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" event={"ID":"094e442e-e861-4888-b314-9c6233b4b321","Type":"ContainerStarted","Data":"bcc17942853c7d2ec2123411528c40638145d715e4252ab4c3c2099a62c4133b"} Sep 13 19:14:35.718026 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:35.717613 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" Sep 13 19:14:35.719987 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:35.719917 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" event={"ID":"698395ff-2ac6-4085-98b7-d702909cfff6","Type":"ContainerStarted","Data":"d2ac41853c7e1c6bae1175fbda254d58ba81b85bcabae04b8615656f9f3acc38"} Sep 13 19:14:35.720199 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:35.720167 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" Sep 13 19:14:35.734707 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:35.734643 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" podStartSLOduration=2.894421652 podStartE2EDuration="5.73462135s" totalImagesPullingTime="2.840199698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:14:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:14:35.733929929 +0000 UTC m=+627.115458957" watchObservedRunningTime="2026-09-13 19:14:35.73462135 +0000 UTC m=+627.116150382" Sep 13 19:14:35.750398 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:35.750342 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" podStartSLOduration=2.311823981 podStartE2EDuration="5.750324024s" totalImagesPullingTime="3.438500043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:14:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:14:35.749963638 +0000 UTC m=+627.131492667" watchObservedRunningTime="2026-09-13 19:14:35.750324024 +0000 UTC m=+627.131853052" Sep 13 19:14:36.666758 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:14:36.666723 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-6gffv" Sep 13 19:15:06.727400 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:06.727364 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f4xt4" Sep 13 19:15:06.730473 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:06.730449 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-98n7v" Sep 13 19:15:38.742386 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:38.742340 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-clzqv"] Sep 13 19:15:38.742869 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:38.742721 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" containerName="discovery" Sep 13 19:15:38.742869 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:38.742733 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" containerName="discovery" Sep 13 19:15:38.742869 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:38.742797 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b163b61-5ab2-4156-9f0e-6fe2d8bfc84a" containerName="discovery" Sep 13 19:15:38.744841 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:38.744822 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-clzqv" Sep 13 19:15:38.747397 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:38.747370 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 13 19:15:38.747526 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:38.747458 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 13 19:15:38.748630 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:38.748609 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-tzdww\"" Sep 13 19:15:38.769400 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:38.769365 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-clzqv"] Sep 13 19:15:38.856692 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:38.856637 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkjgd\" (UniqueName: \"kubernetes.io/projected/cf7fa383-b652-4a0a-9aec-c871900652a5-kube-api-access-dkjgd\") pod \"authorino-operator-76b478bd78-clzqv\" (UID: \"cf7fa383-b652-4a0a-9aec-c871900652a5\") " pod="kuadrant-system/authorino-operator-76b478bd78-clzqv" Sep 13 19:15:38.963969 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:38.958852 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkjgd\" (UniqueName: \"kubernetes.io/projected/cf7fa383-b652-4a0a-9aec-c871900652a5-kube-api-access-dkjgd\") pod \"authorino-operator-76b478bd78-clzqv\" (UID: \"cf7fa383-b652-4a0a-9aec-c871900652a5\") " pod="kuadrant-system/authorino-operator-76b478bd78-clzqv" Sep 13 19:15:38.970146 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:38.970115 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkjgd\" (UniqueName: \"kubernetes.io/projected/cf7fa383-b652-4a0a-9aec-c871900652a5-kube-api-access-dkjgd\") pod \"authorino-operator-76b478bd78-clzqv\" (UID: \"cf7fa383-b652-4a0a-9aec-c871900652a5\") " pod="kuadrant-system/authorino-operator-76b478bd78-clzqv" Sep 13 19:15:39.054659 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:39.054569 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-clzqv" Sep 13 19:15:39.327191 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:39.327161 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-clzqv"] Sep 13 19:15:39.329400 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:15:39.329371 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7fa383_b652_4a0a_9aec_c871900652a5.slice/crio-5bc1847fc7fddc15be5969e719d4245e50c46f70706345e322ea8c1a1091a32c WatchSource:0}: Error finding container 5bc1847fc7fddc15be5969e719d4245e50c46f70706345e322ea8c1a1091a32c: Status 404 returned error can't find the container with id 5bc1847fc7fddc15be5969e719d4245e50c46f70706345e322ea8c1a1091a32c Sep 13 19:15:39.331916 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:39.331897 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 13 19:15:40.030691 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:40.030656 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-clzqv" event={"ID":"cf7fa383-b652-4a0a-9aec-c871900652a5","Type":"ContainerStarted","Data":"5bc1847fc7fddc15be5969e719d4245e50c46f70706345e322ea8c1a1091a32c"} Sep 13 19:15:41.772163 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:41.772124 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln"] Sep 13 19:15:41.774858 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:41.774828 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:15:41.777150 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:41.777124 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-ls9gt\"" Sep 13 19:15:41.785858 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:41.785828 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln"] Sep 13 19:15:41.885742 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:41.885702 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znj2m\" (UniqueName: \"kubernetes.io/projected/e5468f42-b05d-4298-b0e0-9371e687d87d-kube-api-access-znj2m\") pod \"limitador-operator-controller-manager-74495c69f8-zkhln\" (UID: \"e5468f42-b05d-4298-b0e0-9371e687d87d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:15:41.987035 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:41.986985 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znj2m\" (UniqueName: \"kubernetes.io/projected/e5468f42-b05d-4298-b0e0-9371e687d87d-kube-api-access-znj2m\") pod \"limitador-operator-controller-manager-74495c69f8-zkhln\" (UID: \"e5468f42-b05d-4298-b0e0-9371e687d87d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:15:41.998973 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:41.998939 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znj2m\" (UniqueName: \"kubernetes.io/projected/e5468f42-b05d-4298-b0e0-9371e687d87d-kube-api-access-znj2m\") pod \"limitador-operator-controller-manager-74495c69f8-zkhln\" (UID: \"e5468f42-b05d-4298-b0e0-9371e687d87d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:15:42.089156 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:42.089126 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:15:42.366115 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:42.366088 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln"] Sep 13 19:15:42.368708 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:15:42.368669 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5468f42_b05d_4298_b0e0_9371e687d87d.slice/crio-eb74a2ca9b976dea28e069341e025da1ebb40cd0e32ec480ee4cea632c4148ca WatchSource:0}: Error finding container eb74a2ca9b976dea28e069341e025da1ebb40cd0e32ec480ee4cea632c4148ca: Status 404 returned error can't find the container with id eb74a2ca9b976dea28e069341e025da1ebb40cd0e32ec480ee4cea632c4148ca Sep 13 19:15:43.047496 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:43.047453 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-clzqv" event={"ID":"cf7fa383-b652-4a0a-9aec-c871900652a5","Type":"ContainerStarted","Data":"3a2be09f7c6263c1ffc0dcf82df4ff6035c3df2c6f6bad74fb9bbbd9380ed34c"} Sep 13 19:15:43.047902 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:43.047539 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-clzqv" Sep 13 19:15:43.048889 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:43.048859 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" event={"ID":"e5468f42-b05d-4298-b0e0-9371e687d87d","Type":"ContainerStarted","Data":"eb74a2ca9b976dea28e069341e025da1ebb40cd0e32ec480ee4cea632c4148ca"} Sep 13 19:15:43.063991 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:43.063934 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-clzqv" podStartSLOduration=2.349899722 podStartE2EDuration="5.063914966s" totalImagesPullingTime="2.714015244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:15:43.062325184 +0000 UTC m=+694.443854213" watchObservedRunningTime="2026-09-13 19:15:43.063914966 +0000 UTC m=+694.445443995" Sep 13 19:15:44.835217 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:44.835178 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq"] Sep 13 19:15:44.837832 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:44.837805 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq" Sep 13 19:15:44.841682 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:44.841659 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 13 19:15:44.842076 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:44.842057 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-p7v89\"" Sep 13 19:15:44.881489 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:44.881452 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq"] Sep 13 19:15:44.915737 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:44.915701 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvs9q\" (UniqueName: \"kubernetes.io/projected/b44afc5d-e7a1-4c18-a155-42eb4f466ba0-kube-api-access-dvs9q\") pod \"dns-operator-controller-manager-7b84cd7c9c-h5bgq\" (UID: \"b44afc5d-e7a1-4c18-a155-42eb4f466ba0\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq" Sep 13 19:15:45.016947 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:45.016906 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvs9q\" (UniqueName: \"kubernetes.io/projected/b44afc5d-e7a1-4c18-a155-42eb4f466ba0-kube-api-access-dvs9q\") pod \"dns-operator-controller-manager-7b84cd7c9c-h5bgq\" (UID: \"b44afc5d-e7a1-4c18-a155-42eb4f466ba0\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq" Sep 13 19:15:45.028705 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:45.028681 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvs9q\" (UniqueName: \"kubernetes.io/projected/b44afc5d-e7a1-4c18-a155-42eb4f466ba0-kube-api-access-dvs9q\") pod \"dns-operator-controller-manager-7b84cd7c9c-h5bgq\" (UID: \"b44afc5d-e7a1-4c18-a155-42eb4f466ba0\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq" Sep 13 19:15:45.061074 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:45.061036 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" event={"ID":"e5468f42-b05d-4298-b0e0-9371e687d87d","Type":"ContainerStarted","Data":"f473ce973506a5732835407af425d36e469c3b417d87452d8e44575fa8be5273"} Sep 13 19:15:45.061292 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:45.061197 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:15:45.080679 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:45.080623 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" podStartSLOduration=2.081479774 podStartE2EDuration="4.080605128s" totalImagesPullingTime="1.999125354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:15:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:15:45.079589509 +0000 UTC m=+696.461118544" watchObservedRunningTime="2026-09-13 19:15:45.080605128 +0000 UTC m=+696.462134156" Sep 13 19:15:45.148602 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:45.148512 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq" Sep 13 19:15:45.433818 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:45.433527 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq"] Sep 13 19:15:45.436940 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:15:45.436911 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb44afc5d_e7a1_4c18_a155_42eb4f466ba0.slice/crio-8d84dfcbd5dca9e38c894603e57a0b2632781ca58aea6ea98289f0f73dac0d59 WatchSource:0}: Error finding container 8d84dfcbd5dca9e38c894603e57a0b2632781ca58aea6ea98289f0f73dac0d59: Status 404 returned error can't find the container with id 8d84dfcbd5dca9e38c894603e57a0b2632781ca58aea6ea98289f0f73dac0d59 Sep 13 19:15:46.067936 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:46.067899 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq" event={"ID":"b44afc5d-e7a1-4c18-a155-42eb4f466ba0","Type":"ContainerStarted","Data":"8d84dfcbd5dca9e38c894603e57a0b2632781ca58aea6ea98289f0f73dac0d59"} Sep 13 19:15:48.083878 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:48.083840 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq" event={"ID":"b44afc5d-e7a1-4c18-a155-42eb4f466ba0","Type":"ContainerStarted","Data":"d48064fc5ca45e5f0e284a659e3e9456590b68ff4da456bb1f473034484ca65f"} Sep 13 19:15:48.084298 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:48.083923 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq" Sep 13 19:15:48.108671 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:48.108612 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq" podStartSLOduration=2.540303114 podStartE2EDuration="4.108593353s" totalImagesPullingTime="1.568290239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:15:48.106892404 +0000 UTC m=+699.488421448" watchObservedRunningTime="2026-09-13 19:15:48.108593353 +0000 UTC m=+699.490122382" Sep 13 19:15:50.643566 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:50.643514 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j"] Sep 13 19:15:50.646127 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:50.646109 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:15:50.648748 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:50.648712 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q82bf\"" Sep 13 19:15:50.657537 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:50.657507 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j"] Sep 13 19:15:50.771861 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:50.771815 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8919ff1-9380-43aa-bbe8-55eb5402c48d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" (UID: \"e8919ff1-9380-43aa-bbe8-55eb5402c48d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:15:50.772069 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:50.771870 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbslm\" (UniqueName: \"kubernetes.io/projected/e8919ff1-9380-43aa-bbe8-55eb5402c48d-kube-api-access-jbslm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" (UID: \"e8919ff1-9380-43aa-bbe8-55eb5402c48d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:15:50.873169 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:50.873124 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8919ff1-9380-43aa-bbe8-55eb5402c48d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" (UID: \"e8919ff1-9380-43aa-bbe8-55eb5402c48d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:15:50.873384 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:50.873183 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbslm\" (UniqueName: \"kubernetes.io/projected/e8919ff1-9380-43aa-bbe8-55eb5402c48d-kube-api-access-jbslm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" (UID: \"e8919ff1-9380-43aa-bbe8-55eb5402c48d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:15:50.873553 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:50.873529 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8919ff1-9380-43aa-bbe8-55eb5402c48d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" (UID: \"e8919ff1-9380-43aa-bbe8-55eb5402c48d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:15:50.881997 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:50.881965 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbslm\" (UniqueName: \"kubernetes.io/projected/e8919ff1-9380-43aa-bbe8-55eb5402c48d-kube-api-access-jbslm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" (UID: \"e8919ff1-9380-43aa-bbe8-55eb5402c48d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:15:50.957784 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:50.957694 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:15:51.236659 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:51.236623 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j"] Sep 13 19:15:51.242182 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:15:51.242152 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8919ff1_9380_43aa_bbe8_55eb5402c48d.slice/crio-937cff231964ab83cafb5e288db04dd412cbdc3e8097c3b43fa75e90bfaa6ec7 WatchSource:0}: Error finding container 937cff231964ab83cafb5e288db04dd412cbdc3e8097c3b43fa75e90bfaa6ec7: Status 404 returned error can't find the container with id 937cff231964ab83cafb5e288db04dd412cbdc3e8097c3b43fa75e90bfaa6ec7 Sep 13 19:15:52.104297 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:52.104263 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" event={"ID":"e8919ff1-9380-43aa-bbe8-55eb5402c48d","Type":"ContainerStarted","Data":"937cff231964ab83cafb5e288db04dd412cbdc3e8097c3b43fa75e90bfaa6ec7"} Sep 13 19:15:54.056513 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:54.056471 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-clzqv" Sep 13 19:15:56.070820 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:56.070783 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:15:56.132731 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:56.132696 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" event={"ID":"e8919ff1-9380-43aa-bbe8-55eb5402c48d","Type":"ContainerStarted","Data":"4f8f3c40c9d7dd40d40ed5f29013d6569c10a7185470c91eea6a933bbf7f4f5b"} Sep 13 19:15:56.132933 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:56.132829 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:15:56.150185 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:56.150125 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" podStartSLOduration=2.289942941 podStartE2EDuration="6.150104899s" totalImagesPullingTime="3.860161958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:15:56.149003104 +0000 UTC m=+707.530532134" watchObservedRunningTime="2026-09-13 19:15:56.150104899 +0000 UTC m=+707.531633928" Sep 13 19:15:57.775107 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:57.775071 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4"] Sep 13 19:15:57.777809 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:57.777786 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" Sep 13 19:15:57.780076 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:57.780052 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 13 19:15:57.780469 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:57.780448 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 13 19:15:57.780568 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:57.780448 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-95phb\"" Sep 13 19:15:57.786040 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:57.785893 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4"] Sep 13 19:15:57.940355 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:57.940318 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a18120ff-b7bb-4165-bd40-34de6d76dda8-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-7h6x4\" (UID: \"a18120ff-b7bb-4165-bd40-34de6d76dda8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" Sep 13 19:15:57.940533 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:57.940472 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a18120ff-b7bb-4165-bd40-34de6d76dda8-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-7h6x4\" (UID: \"a18120ff-b7bb-4165-bd40-34de6d76dda8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" Sep 13 19:15:57.940624 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:57.940539 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fft97\" (UniqueName: \"kubernetes.io/projected/a18120ff-b7bb-4165-bd40-34de6d76dda8-kube-api-access-fft97\") pod \"kuadrant-console-plugin-fbdd4f445-7h6x4\" (UID: \"a18120ff-b7bb-4165-bd40-34de6d76dda8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" Sep 13 19:15:58.041537 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:58.041443 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fft97\" (UniqueName: \"kubernetes.io/projected/a18120ff-b7bb-4165-bd40-34de6d76dda8-kube-api-access-fft97\") pod \"kuadrant-console-plugin-fbdd4f445-7h6x4\" (UID: \"a18120ff-b7bb-4165-bd40-34de6d76dda8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" Sep 13 19:15:58.041537 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:58.041522 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a18120ff-b7bb-4165-bd40-34de6d76dda8-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-7h6x4\" (UID: \"a18120ff-b7bb-4165-bd40-34de6d76dda8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" Sep 13 19:15:58.041736 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:58.041668 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a18120ff-b7bb-4165-bd40-34de6d76dda8-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-7h6x4\" (UID: \"a18120ff-b7bb-4165-bd40-34de6d76dda8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" Sep 13 19:15:58.041854 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:15:58.041835 2563 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Sep 13 19:15:58.041926 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:15:58.041915 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a18120ff-b7bb-4165-bd40-34de6d76dda8-plugin-serving-cert podName:a18120ff-b7bb-4165-bd40-34de6d76dda8 nodeName:}" failed. No retries permitted until 2026-09-13 19:15:58.541893923 +0000 UTC m=+709.923422947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/a18120ff-b7bb-4165-bd40-34de6d76dda8-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-7h6x4" (UID: "a18120ff-b7bb-4165-bd40-34de6d76dda8") : secret "plugin-serving-cert" not found Sep 13 19:15:58.042215 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:58.042193 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a18120ff-b7bb-4165-bd40-34de6d76dda8-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-7h6x4\" (UID: \"a18120ff-b7bb-4165-bd40-34de6d76dda8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" Sep 13 19:15:58.052156 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:58.052118 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fft97\" (UniqueName: \"kubernetes.io/projected/a18120ff-b7bb-4165-bd40-34de6d76dda8-kube-api-access-fft97\") pod \"kuadrant-console-plugin-fbdd4f445-7h6x4\" (UID: \"a18120ff-b7bb-4165-bd40-34de6d76dda8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" Sep 13 19:15:58.546667 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:58.546628 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a18120ff-b7bb-4165-bd40-34de6d76dda8-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-7h6x4\" (UID: \"a18120ff-b7bb-4165-bd40-34de6d76dda8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" Sep 13 19:15:58.549199 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:58.549168 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a18120ff-b7bb-4165-bd40-34de6d76dda8-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-7h6x4\" (UID: \"a18120ff-b7bb-4165-bd40-34de6d76dda8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" Sep 13 19:15:58.689150 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:58.689109 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" Sep 13 19:15:58.959645 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:58.959614 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4"] Sep 13 19:15:58.961471 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:15:58.961442 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda18120ff_b7bb_4165_bd40_34de6d76dda8.slice/crio-f2b4683d615c993b78eb5f17c7cf71d47d31313ca80eab6c83ce10409827321e WatchSource:0}: Error finding container f2b4683d615c993b78eb5f17c7cf71d47d31313ca80eab6c83ce10409827321e: Status 404 returned error can't find the container with id f2b4683d615c993b78eb5f17c7cf71d47d31313ca80eab6c83ce10409827321e Sep 13 19:15:59.091595 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:59.091565 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-h5bgq" Sep 13 19:15:59.152176 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:15:59.152136 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" event={"ID":"a18120ff-b7bb-4165-bd40-34de6d76dda8","Type":"ContainerStarted","Data":"f2b4683d615c993b78eb5f17c7cf71d47d31313ca80eab6c83ce10409827321e"} Sep 13 19:16:04.181830 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:04.181791 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" event={"ID":"a18120ff-b7bb-4165-bd40-34de6d76dda8","Type":"ContainerStarted","Data":"803f8b60d70170600ff285ffd749ee82f55f18b58d23bc7717fdfb513cd2db11"} Sep 13 19:16:04.200498 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:04.200437 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-7h6x4" podStartSLOduration=2.7039886490000002 podStartE2EDuration="7.200409902s" totalImagesPullingTime="4.496421253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:15:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:16:04.197925803 +0000 UTC m=+715.579454833" watchObservedRunningTime="2026-09-13 19:16:04.200409902 +0000 UTC m=+715.581938931" Sep 13 19:16:07.143009 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:07.142971 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:16:09.183989 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.183947 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j"] Sep 13 19:16:09.184506 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.184163 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" podUID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" containerName="manager" containerID="cri-o://4f8f3c40c9d7dd40d40ed5f29013d6569c10a7185470c91eea6a933bbf7f4f5b" gracePeriod=2 Sep 13 19:16:09.200829 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.200789 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf"] Sep 13 19:16:09.204790 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.204761 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j"] Sep 13 19:16:09.204920 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.204862 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" Sep 13 19:16:09.206271 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:09.206153 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e8919ff1-9380-43aa-bbe8-55eb5402c48d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-tct6j\": pods \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:16:09.208384 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:09.208349 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:16:09.213336 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.213306 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln"] Sep 13 19:16:09.213558 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.213535 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" podUID="e5468f42-b05d-4298-b0e0-9371e687d87d" containerName="manager" containerID="cri-o://f473ce973506a5732835407af425d36e469c3b417d87452d8e44575fa8be5273" gracePeriod=2 Sep 13 19:16:09.216291 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.216261 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb"] Sep 13 19:16:09.216826 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.216804 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" containerName="manager" Sep 13 19:16:09.216826 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.216826 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" containerName="manager" Sep 13 19:16:09.217144 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.216921 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" containerName="manager" Sep 13 19:16:09.219305 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.219280 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf"] Sep 13 19:16:09.219415 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.219401 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" Sep 13 19:16:09.231737 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.231706 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln"] Sep 13 19:16:09.236183 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.236149 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb"] Sep 13 19:16:09.239468 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:09.239394 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zkhln\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e5468f42-b05d-4298-b0e0-9371e687d87d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:16:09.241800 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:09.241754 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:16:09.243002 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.242972 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69"] Sep 13 19:16:09.243839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.243821 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5468f42-b05d-4298-b0e0-9371e687d87d" containerName="manager" Sep 13 19:16:09.243839 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.243839 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5468f42-b05d-4298-b0e0-9371e687d87d" containerName="manager" Sep 13 19:16:09.243954 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.243917 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5468f42-b05d-4298-b0e0-9371e687d87d" containerName="manager" Sep 13 19:16:09.245869 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.245855 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69" Sep 13 19:16:09.252491 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.252460 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szl5l\" (UniqueName: \"kubernetes.io/projected/6689cd67-1c0f-44ea-8149-76859b424e83-kube-api-access-szl5l\") pod \"limitador-operator-controller-manager-74495c69f8-76p69\" (UID: \"6689cd67-1c0f-44ea-8149-76859b424e83\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69" Sep 13 19:16:09.257574 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.257547 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69"] Sep 13 19:16:09.265407 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:09.265362 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zkhln\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e5468f42-b05d-4298-b0e0-9371e687d87d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:16:09.267568 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:09.267529 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:16:09.354436 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.354166 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d3071f9-dce8-4e1f-b158-51ca221002e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vw2sb\" (UID: \"6d3071f9-dce8-4e1f-b158-51ca221002e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" Sep 13 19:16:09.354436 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.354290 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vmmf\" (UniqueName: \"kubernetes.io/projected/5b25ae32-03c4-4a92-a39e-5987ec17c7f5-kube-api-access-7vmmf\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vkvzf\" (UID: \"5b25ae32-03c4-4a92-a39e-5987ec17c7f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" Sep 13 19:16:09.354436 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.354335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl867\" (UniqueName: \"kubernetes.io/projected/6d3071f9-dce8-4e1f-b158-51ca221002e6-kube-api-access-pl867\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vw2sb\" (UID: \"6d3071f9-dce8-4e1f-b158-51ca221002e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" Sep 13 19:16:09.354436 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.354380 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szl5l\" (UniqueName: \"kubernetes.io/projected/6689cd67-1c0f-44ea-8149-76859b424e83-kube-api-access-szl5l\") pod \"limitador-operator-controller-manager-74495c69f8-76p69\" (UID: \"6689cd67-1c0f-44ea-8149-76859b424e83\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69" Sep 13 19:16:09.354436 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.354437 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b25ae32-03c4-4a92-a39e-5987ec17c7f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vkvzf\" (UID: \"5b25ae32-03c4-4a92-a39e-5987ec17c7f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" Sep 13 19:16:09.363710 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.363632 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szl5l\" (UniqueName: \"kubernetes.io/projected/6689cd67-1c0f-44ea-8149-76859b424e83-kube-api-access-szl5l\") pod \"limitador-operator-controller-manager-74495c69f8-76p69\" (UID: \"6689cd67-1c0f-44ea-8149-76859b424e83\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69" Sep 13 19:16:09.454842 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.454816 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vmmf\" (UniqueName: \"kubernetes.io/projected/5b25ae32-03c4-4a92-a39e-5987ec17c7f5-kube-api-access-7vmmf\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vkvzf\" (UID: \"5b25ae32-03c4-4a92-a39e-5987ec17c7f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" Sep 13 19:16:09.455002 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.454864 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl867\" (UniqueName: \"kubernetes.io/projected/6d3071f9-dce8-4e1f-b158-51ca221002e6-kube-api-access-pl867\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vw2sb\" (UID: \"6d3071f9-dce8-4e1f-b158-51ca221002e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" Sep 13 19:16:09.455002 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.454925 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b25ae32-03c4-4a92-a39e-5987ec17c7f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vkvzf\" (UID: \"5b25ae32-03c4-4a92-a39e-5987ec17c7f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" Sep 13 19:16:09.455002 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.454972 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d3071f9-dce8-4e1f-b158-51ca221002e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vw2sb\" (UID: \"6d3071f9-dce8-4e1f-b158-51ca221002e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" Sep 13 19:16:09.455373 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.455346 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b25ae32-03c4-4a92-a39e-5987ec17c7f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vkvzf\" (UID: \"5b25ae32-03c4-4a92-a39e-5987ec17c7f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" Sep 13 19:16:09.455485 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.455396 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d3071f9-dce8-4e1f-b158-51ca221002e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vw2sb\" (UID: \"6d3071f9-dce8-4e1f-b158-51ca221002e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" Sep 13 19:16:09.457720 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.457697 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:16:09.460001 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:09.459973 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zkhln\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e5468f42-b05d-4298-b0e0-9371e687d87d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:16:09.460968 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.460950 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:16:09.461985 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:09.461954 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:16:09.463797 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:09.463761 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zkhln\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e5468f42-b05d-4298-b0e0-9371e687d87d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:16:09.464394 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.464373 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl867\" (UniqueName: \"kubernetes.io/projected/6d3071f9-dce8-4e1f-b158-51ca221002e6-kube-api-access-pl867\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vw2sb\" (UID: \"6d3071f9-dce8-4e1f-b158-51ca221002e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" Sep 13 19:16:09.464491 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.464390 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vmmf\" (UniqueName: \"kubernetes.io/projected/5b25ae32-03c4-4a92-a39e-5987ec17c7f5-kube-api-access-7vmmf\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vkvzf\" (UID: \"5b25ae32-03c4-4a92-a39e-5987ec17c7f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" Sep 13 19:16:09.465561 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:09.465536 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:16:09.555950 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.555913 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znj2m\" (UniqueName: \"kubernetes.io/projected/e5468f42-b05d-4298-b0e0-9371e687d87d-kube-api-access-znj2m\") pod \"e5468f42-b05d-4298-b0e0-9371e687d87d\" (UID: \"e5468f42-b05d-4298-b0e0-9371e687d87d\") " Sep 13 19:16:09.555950 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.555954 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbslm\" (UniqueName: \"kubernetes.io/projected/e8919ff1-9380-43aa-bbe8-55eb5402c48d-kube-api-access-jbslm\") pod \"e8919ff1-9380-43aa-bbe8-55eb5402c48d\" (UID: \"e8919ff1-9380-43aa-bbe8-55eb5402c48d\") " Sep 13 19:16:09.556190 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.556005 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8919ff1-9380-43aa-bbe8-55eb5402c48d-extensions-socket-volume\") pod \"e8919ff1-9380-43aa-bbe8-55eb5402c48d\" (UID: \"e8919ff1-9380-43aa-bbe8-55eb5402c48d\") " Sep 13 19:16:09.556497 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.556465 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8919ff1-9380-43aa-bbe8-55eb5402c48d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e8919ff1-9380-43aa-bbe8-55eb5402c48d" (UID: "e8919ff1-9380-43aa-bbe8-55eb5402c48d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 19:16:09.558339 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.558313 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5468f42-b05d-4298-b0e0-9371e687d87d-kube-api-access-znj2m" (OuterVolumeSpecName: "kube-api-access-znj2m") pod "e5468f42-b05d-4298-b0e0-9371e687d87d" (UID: "e5468f42-b05d-4298-b0e0-9371e687d87d"). InnerVolumeSpecName "kube-api-access-znj2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:16:09.558414 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.558350 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8919ff1-9380-43aa-bbe8-55eb5402c48d-kube-api-access-jbslm" (OuterVolumeSpecName: "kube-api-access-jbslm") pod "e8919ff1-9380-43aa-bbe8-55eb5402c48d" (UID: "e8919ff1-9380-43aa-bbe8-55eb5402c48d"). InnerVolumeSpecName "kube-api-access-jbslm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:16:09.604980 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.604940 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" Sep 13 19:16:09.611855 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.611813 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" Sep 13 19:16:09.617820 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.617788 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69" Sep 13 19:16:09.658786 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.657863 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8919ff1-9380-43aa-bbe8-55eb5402c48d-extensions-socket-volume\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:16:09.658786 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.657910 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znj2m\" (UniqueName: \"kubernetes.io/projected/e5468f42-b05d-4298-b0e0-9371e687d87d-kube-api-access-znj2m\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:16:09.658786 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.657931 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbslm\" (UniqueName: \"kubernetes.io/projected/e8919ff1-9380-43aa-bbe8-55eb5402c48d-kube-api-access-jbslm\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:16:09.892751 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.892722 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf"] Sep 13 19:16:09.894018 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:16:09.893990 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b25ae32_03c4_4a92_a39e_5987ec17c7f5.slice/crio-04b8b238b1d2799fea0728907f1c5f907dd5ae85e0e899527c22cb92c41f8425 WatchSource:0}: Error finding container 04b8b238b1d2799fea0728907f1c5f907dd5ae85e0e899527c22cb92c41f8425: Status 404 returned error can't find the container with id 04b8b238b1d2799fea0728907f1c5f907dd5ae85e0e899527c22cb92c41f8425 Sep 13 19:16:09.910172 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.910147 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb"] Sep 13 19:16:09.912925 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:09.912899 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69"] Sep 13 19:16:09.914013 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:16:09.913988 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6689cd67_1c0f_44ea_8149_76859b424e83.slice/crio-c19ed556dcac44102cd4e6010e1e4bd3bd7750a1d1577c7afffdda88a4aa2ed9 WatchSource:0}: Error finding container c19ed556dcac44102cd4e6010e1e4bd3bd7750a1d1577c7afffdda88a4aa2ed9: Status 404 returned error can't find the container with id c19ed556dcac44102cd4e6010e1e4bd3bd7750a1d1577c7afffdda88a4aa2ed9 Sep 13 19:16:09.914854 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:16:09.914776 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d3071f9_dce8_4e1f_b158_51ca221002e6.slice/crio-0db092e60c4258d6a8e38c5cc8cbd3dfa31547316b0f2b331a4a8dfd3c69d6bf WatchSource:0}: Error finding container 0db092e60c4258d6a8e38c5cc8cbd3dfa31547316b0f2b331a4a8dfd3c69d6bf: Status 404 returned error can't find the container with id 0db092e60c4258d6a8e38c5cc8cbd3dfa31547316b0f2b331a4a8dfd3c69d6bf Sep 13 19:16:10.219007 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.218967 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69" event={"ID":"6689cd67-1c0f-44ea-8149-76859b424e83","Type":"ContainerStarted","Data":"27eaa9adc548a4f6730a778903aab2a407bd9cdeac60b91c6fa20e56baa784ab"} Sep 13 19:16:10.219483 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.219036 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69" event={"ID":"6689cd67-1c0f-44ea-8149-76859b424e83","Type":"ContainerStarted","Data":"c19ed556dcac44102cd4e6010e1e4bd3bd7750a1d1577c7afffdda88a4aa2ed9"} Sep 13 19:16:10.219483 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.219063 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69" Sep 13 19:16:10.220883 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.220857 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" event={"ID":"5b25ae32-03c4-4a92-a39e-5987ec17c7f5","Type":"ContainerStarted","Data":"4b219488373be0e4a4d3fa8a87e2308c1fd1363d26eb80cf2664dcf8b1514a2c"} Sep 13 19:16:10.220994 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.220904 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" event={"ID":"5b25ae32-03c4-4a92-a39e-5987ec17c7f5","Type":"ContainerStarted","Data":"04b8b238b1d2799fea0728907f1c5f907dd5ae85e0e899527c22cb92c41f8425"} Sep 13 19:16:10.220994 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.220957 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" Sep 13 19:16:10.221495 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:10.221470 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zkhln\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e5468f42-b05d-4298-b0e0-9371e687d87d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:16:10.222384 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.222360 2563 generic.go:356] "Generic (PLEG): container finished" podID="e5468f42-b05d-4298-b0e0-9371e687d87d" containerID="f473ce973506a5732835407af425d36e469c3b417d87452d8e44575fa8be5273" exitCode=0 Sep 13 19:16:10.222489 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.222407 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:16:10.222489 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.222439 2563 scope.go:117] "RemoveContainer" containerID="f473ce973506a5732835407af425d36e469c3b417d87452d8e44575fa8be5273" Sep 13 19:16:10.223846 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:10.223580 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:16:10.224431 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.224403 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" event={"ID":"6d3071f9-dce8-4e1f-b158-51ca221002e6","Type":"ContainerStarted","Data":"f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047"} Sep 13 19:16:10.224700 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.224462 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" event={"ID":"6d3071f9-dce8-4e1f-b158-51ca221002e6","Type":"ContainerStarted","Data":"0db092e60c4258d6a8e38c5cc8cbd3dfa31547316b0f2b331a4a8dfd3c69d6bf"} Sep 13 19:16:10.224700 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.224495 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" Sep 13 19:16:10.226127 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.226103 2563 generic.go:356] "Generic (PLEG): container finished" podID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" containerID="4f8f3c40c9d7dd40d40ed5f29013d6569c10a7185470c91eea6a933bbf7f4f5b" exitCode=0 Sep 13 19:16:10.226220 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.226137 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:16:10.232630 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.232604 2563 scope.go:117] "RemoveContainer" containerID="f473ce973506a5732835407af425d36e469c3b417d87452d8e44575fa8be5273" Sep 13 19:16:10.232931 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:10.232910 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f473ce973506a5732835407af425d36e469c3b417d87452d8e44575fa8be5273\": container with ID starting with f473ce973506a5732835407af425d36e469c3b417d87452d8e44575fa8be5273 not found: ID does not exist" containerID="f473ce973506a5732835407af425d36e469c3b417d87452d8e44575fa8be5273" Sep 13 19:16:10.233003 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.232939 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f473ce973506a5732835407af425d36e469c3b417d87452d8e44575fa8be5273"} err="failed to get container status \"f473ce973506a5732835407af425d36e469c3b417d87452d8e44575fa8be5273\": rpc error: code = NotFound desc = could not find container \"f473ce973506a5732835407af425d36e469c3b417d87452d8e44575fa8be5273\": container with ID starting with f473ce973506a5732835407af425d36e469c3b417d87452d8e44575fa8be5273 not found: ID does not exist" Sep 13 19:16:10.233003 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.232957 2563 scope.go:117] "RemoveContainer" containerID="4f8f3c40c9d7dd40d40ed5f29013d6569c10a7185470c91eea6a933bbf7f4f5b" Sep 13 19:16:10.242522 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.242416 2563 scope.go:117] "RemoveContainer" containerID="4f8f3c40c9d7dd40d40ed5f29013d6569c10a7185470c91eea6a933bbf7f4f5b" Sep 13 19:16:10.243730 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:10.243142 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f8f3c40c9d7dd40d40ed5f29013d6569c10a7185470c91eea6a933bbf7f4f5b\": container with ID starting with 4f8f3c40c9d7dd40d40ed5f29013d6569c10a7185470c91eea6a933bbf7f4f5b not found: ID does not exist" containerID="4f8f3c40c9d7dd40d40ed5f29013d6569c10a7185470c91eea6a933bbf7f4f5b" Sep 13 19:16:10.243730 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.243181 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f8f3c40c9d7dd40d40ed5f29013d6569c10a7185470c91eea6a933bbf7f4f5b"} err="failed to get container status \"4f8f3c40c9d7dd40d40ed5f29013d6569c10a7185470c91eea6a933bbf7f4f5b\": rpc error: code = NotFound desc = could not find container \"4f8f3c40c9d7dd40d40ed5f29013d6569c10a7185470c91eea6a933bbf7f4f5b\": container with ID starting with 4f8f3c40c9d7dd40d40ed5f29013d6569c10a7185470c91eea6a933bbf7f4f5b not found: ID does not exist" Sep 13 19:16:10.244579 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:10.244548 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zkhln\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e5468f42-b05d-4298-b0e0-9371e687d87d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zkhln" Sep 13 19:16:10.244763 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.244724 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69" podStartSLOduration=1.24470879 podStartE2EDuration="1.24470879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:16:10.242388466 +0000 UTC m=+721.623917495" watchObservedRunningTime="2026-09-13 19:16:10.24470879 +0000 UTC m=+721.626237820" Sep 13 19:16:10.246522 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:10.246484 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-tct6j\" is forbidden: User \"system:node:ip-10-0-129-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-160.ec2.internal' and this object" podUID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tct6j" Sep 13 19:16:10.268555 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.268476 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" podStartSLOduration=1.268455693 podStartE2EDuration="1.268455693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:16:10.267375218 +0000 UTC m=+721.648904248" watchObservedRunningTime="2026-09-13 19:16:10.268455693 +0000 UTC m=+721.649984723" Sep 13 19:16:10.286343 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:10.286286 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" podStartSLOduration=1.286268672 podStartE2EDuration="1.286268672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:16:10.285178349 +0000 UTC m=+721.666707370" watchObservedRunningTime="2026-09-13 19:16:10.286268672 +0000 UTC m=+721.667797700" Sep 13 19:16:11.133716 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:11.133628 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5468f42-b05d-4298-b0e0-9371e687d87d" path="/var/lib/kubelet/pods/e5468f42-b05d-4298-b0e0-9371e687d87d/volumes" Sep 13 19:16:11.134827 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:11.134672 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8919ff1-9380-43aa-bbe8-55eb5402c48d" path="/var/lib/kubelet/pods/e8919ff1-9380-43aa-bbe8-55eb5402c48d/volumes" Sep 13 19:16:21.235929 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:21.235892 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" Sep 13 19:16:21.236333 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:21.235977 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76p69" Sep 13 19:16:21.236333 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:21.236141 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" Sep 13 19:16:21.319279 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:21.319225 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb"] Sep 13 19:16:21.319562 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:21.319514 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" podUID="6d3071f9-dce8-4e1f-b158-51ca221002e6" containerName="manager" containerID="cri-o://f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047" gracePeriod=10 Sep 13 19:16:22.097750 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.097714 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" Sep 13 19:16:22.169006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.168972 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d3071f9-dce8-4e1f-b158-51ca221002e6-extensions-socket-volume\") pod \"6d3071f9-dce8-4e1f-b158-51ca221002e6\" (UID: \"6d3071f9-dce8-4e1f-b158-51ca221002e6\") " Sep 13 19:16:22.169196 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.169041 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pl867\" (UniqueName: \"kubernetes.io/projected/6d3071f9-dce8-4e1f-b158-51ca221002e6-kube-api-access-pl867\") pod \"6d3071f9-dce8-4e1f-b158-51ca221002e6\" (UID: \"6d3071f9-dce8-4e1f-b158-51ca221002e6\") " Sep 13 19:16:22.169455 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.169427 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d3071f9-dce8-4e1f-b158-51ca221002e6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6d3071f9-dce8-4e1f-b158-51ca221002e6" (UID: "6d3071f9-dce8-4e1f-b158-51ca221002e6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 19:16:22.171191 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.171162 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d3071f9-dce8-4e1f-b158-51ca221002e6-kube-api-access-pl867" (OuterVolumeSpecName: "kube-api-access-pl867") pod "6d3071f9-dce8-4e1f-b158-51ca221002e6" (UID: "6d3071f9-dce8-4e1f-b158-51ca221002e6"). InnerVolumeSpecName "kube-api-access-pl867". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:16:22.270497 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.270456 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d3071f9-dce8-4e1f-b158-51ca221002e6-extensions-socket-volume\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:16:22.270497 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.270490 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pl867\" (UniqueName: \"kubernetes.io/projected/6d3071f9-dce8-4e1f-b158-51ca221002e6-kube-api-access-pl867\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:16:22.294370 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.294334 2563 generic.go:356] "Generic (PLEG): container finished" podID="6d3071f9-dce8-4e1f-b158-51ca221002e6" containerID="f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047" exitCode=0 Sep 13 19:16:22.294509 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.294382 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" Sep 13 19:16:22.294509 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.294393 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" event={"ID":"6d3071f9-dce8-4e1f-b158-51ca221002e6","Type":"ContainerDied","Data":"f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047"} Sep 13 19:16:22.294509 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.294436 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb" event={"ID":"6d3071f9-dce8-4e1f-b158-51ca221002e6","Type":"ContainerDied","Data":"0db092e60c4258d6a8e38c5cc8cbd3dfa31547316b0f2b331a4a8dfd3c69d6bf"} Sep 13 19:16:22.294509 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.294458 2563 scope.go:117] "RemoveContainer" containerID="f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047" Sep 13 19:16:22.303939 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.303918 2563 scope.go:117] "RemoveContainer" containerID="f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047" Sep 13 19:16:22.304283 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:16:22.304260 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047\": container with ID starting with f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047 not found: ID does not exist" containerID="f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047" Sep 13 19:16:22.304348 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.304293 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047"} err="failed to get container status \"f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047\": rpc error: code = NotFound desc = could not find container \"f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047\": container with ID starting with f9278dcc965b8a9853ef4aa3f18feb2f1230b6aaba967a0682493185e2380047 not found: ID does not exist" Sep 13 19:16:22.318493 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.318451 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb"] Sep 13 19:16:22.326523 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:22.326493 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vw2sb"] Sep 13 19:16:23.131393 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:23.131355 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d3071f9-dce8-4e1f-b158-51ca221002e6" path="/var/lib/kubelet/pods/6d3071f9-dce8-4e1f-b158-51ca221002e6/volumes" Sep 13 19:16:34.220515 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:34.220479 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf"] Sep 13 19:16:34.220982 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:34.220716 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" podUID="5b25ae32-03c4-4a92-a39e-5987ec17c7f5" containerName="manager" containerID="cri-o://4b219488373be0e4a4d3fa8a87e2308c1fd1363d26eb80cf2664dcf8b1514a2c" gracePeriod=10 Sep 13 19:16:34.364511 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:34.364469 2563 generic.go:356] "Generic (PLEG): container finished" podID="5b25ae32-03c4-4a92-a39e-5987ec17c7f5" containerID="4b219488373be0e4a4d3fa8a87e2308c1fd1363d26eb80cf2664dcf8b1514a2c" exitCode=0 Sep 13 19:16:34.364662 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:34.364547 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" event={"ID":"5b25ae32-03c4-4a92-a39e-5987ec17c7f5","Type":"ContainerDied","Data":"4b219488373be0e4a4d3fa8a87e2308c1fd1363d26eb80cf2664dcf8b1514a2c"} Sep 13 19:16:34.482052 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:34.481981 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" Sep 13 19:16:34.580022 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:34.579979 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b25ae32-03c4-4a92-a39e-5987ec17c7f5-extensions-socket-volume\") pod \"5b25ae32-03c4-4a92-a39e-5987ec17c7f5\" (UID: \"5b25ae32-03c4-4a92-a39e-5987ec17c7f5\") " Sep 13 19:16:34.580198 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:34.580044 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vmmf\" (UniqueName: \"kubernetes.io/projected/5b25ae32-03c4-4a92-a39e-5987ec17c7f5-kube-api-access-7vmmf\") pod \"5b25ae32-03c4-4a92-a39e-5987ec17c7f5\" (UID: \"5b25ae32-03c4-4a92-a39e-5987ec17c7f5\") " Sep 13 19:16:34.580495 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:34.580462 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b25ae32-03c4-4a92-a39e-5987ec17c7f5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5b25ae32-03c4-4a92-a39e-5987ec17c7f5" (UID: "5b25ae32-03c4-4a92-a39e-5987ec17c7f5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 19:16:34.582227 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:34.582199 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b25ae32-03c4-4a92-a39e-5987ec17c7f5-kube-api-access-7vmmf" (OuterVolumeSpecName: "kube-api-access-7vmmf") pod "5b25ae32-03c4-4a92-a39e-5987ec17c7f5" (UID: "5b25ae32-03c4-4a92-a39e-5987ec17c7f5"). InnerVolumeSpecName "kube-api-access-7vmmf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:16:34.681606 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:34.681559 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b25ae32-03c4-4a92-a39e-5987ec17c7f5-extensions-socket-volume\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:16:34.681606 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:34.681597 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vmmf\" (UniqueName: \"kubernetes.io/projected/5b25ae32-03c4-4a92-a39e-5987ec17c7f5-kube-api-access-7vmmf\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:16:35.371834 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:35.371804 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" Sep 13 19:16:35.372277 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:35.371843 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf" event={"ID":"5b25ae32-03c4-4a92-a39e-5987ec17c7f5","Type":"ContainerDied","Data":"04b8b238b1d2799fea0728907f1c5f907dd5ae85e0e899527c22cb92c41f8425"} Sep 13 19:16:35.372277 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:35.371923 2563 scope.go:117] "RemoveContainer" containerID="4b219488373be0e4a4d3fa8a87e2308c1fd1363d26eb80cf2664dcf8b1514a2c" Sep 13 19:16:35.391399 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:35.391313 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf"] Sep 13 19:16:35.394651 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:35.394621 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vkvzf"] Sep 13 19:16:37.131211 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.131172 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b25ae32-03c4-4a92-a39e-5987ec17c7f5" path="/var/lib/kubelet/pods/5b25ae32-03c4-4a92-a39e-5987ec17c7f5/volumes" Sep 13 19:16:37.769033 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.769000 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd"] Sep 13 19:16:37.769417 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.769402 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b25ae32-03c4-4a92-a39e-5987ec17c7f5" containerName="manager" Sep 13 19:16:37.769417 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.769418 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b25ae32-03c4-4a92-a39e-5987ec17c7f5" containerName="manager" Sep 13 19:16:37.769544 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.769431 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d3071f9-dce8-4e1f-b158-51ca221002e6" containerName="manager" Sep 13 19:16:37.769544 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.769436 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d3071f9-dce8-4e1f-b158-51ca221002e6" containerName="manager" Sep 13 19:16:37.769544 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.769514 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d3071f9-dce8-4e1f-b158-51ca221002e6" containerName="manager" Sep 13 19:16:37.769544 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.769521 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b25ae32-03c4-4a92-a39e-5987ec17c7f5" containerName="manager" Sep 13 19:16:37.772490 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.772465 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:37.775045 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.775019 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-c9dp2\"" Sep 13 19:16:37.782399 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.782365 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd"] Sep 13 19:16:37.913151 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.913114 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:37.913373 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.913172 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:37.913373 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.913271 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:37.913373 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.913306 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sthdx\" (UniqueName: \"kubernetes.io/projected/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-kube-api-access-sthdx\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:37.913373 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.913329 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:37.913373 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.913346 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:37.913564 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.913460 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:37.913564 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.913508 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:37.913564 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:37.913544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.014876 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.014836 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015065 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.014888 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015065 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.014935 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015065 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.014964 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sthdx\" (UniqueName: \"kubernetes.io/projected/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-kube-api-access-sthdx\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015065 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.014992 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015065 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.015017 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015349 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.015071 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015349 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.015105 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015349 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.015134 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015552 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.015492 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015744 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.015701 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015830 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.015771 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015830 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.015812 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.015933 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.015916 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.017713 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.017684 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.017903 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.017882 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.022498 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.022425 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.022660 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.022641 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sthdx\" (UniqueName: \"kubernetes.io/projected/ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb-kube-api-access-sthdx\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b9dvd\" (UID: \"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.085767 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.085721 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:38.365377 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.365341 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd"] Sep 13 19:16:38.367194 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:16:38.367158 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef7b7dc3_2448_4ef9_8198_c6e6b12fb1bb.slice/crio-2430a3d93f83135de7bf0b5bf082d75fdd484289ecdfcdb45a602d1d2b7d61f3 WatchSource:0}: Error finding container 2430a3d93f83135de7bf0b5bf082d75fdd484289ecdfcdb45a602d1d2b7d61f3: Status 404 returned error can't find the container with id 2430a3d93f83135de7bf0b5bf082d75fdd484289ecdfcdb45a602d1d2b7d61f3 Sep 13 19:16:38.369505 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.369465 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 13 19:16:38.369603 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.369543 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 13 19:16:38.369603 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.369575 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 13 19:16:38.391566 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:38.391535 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" event={"ID":"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb","Type":"ContainerStarted","Data":"2430a3d93f83135de7bf0b5bf082d75fdd484289ecdfcdb45a602d1d2b7d61f3"} Sep 13 19:16:39.399311 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:39.399266 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" event={"ID":"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb","Type":"ContainerStarted","Data":"821f8918b1be906936c02a062d672df90757eda3ec5d6e69a7ca23d3e9adc6a1"} Sep 13 19:16:39.421356 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:39.421301 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" podStartSLOduration=2.421274125 podStartE2EDuration="2.421274125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:16:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:16:39.42030191 +0000 UTC m=+750.801830939" watchObservedRunningTime="2026-09-13 19:16:39.421274125 +0000 UTC m=+750.802803153" Sep 13 19:16:40.086838 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:40.086794 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:40.093261 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:40.093204 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:40.405291 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:40.405178 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:40.406182 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:40.406160 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:16:51.913147 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:51.913110 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cgmq7"] Sep 13 19:16:51.915691 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:51.915672 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" Sep 13 19:16:51.917996 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:51.917974 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 13 19:16:51.925096 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:51.925060 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cgmq7"] Sep 13 19:16:52.010442 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.010407 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cgmq7"] Sep 13 19:16:52.055116 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.055078 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srrlr\" (UniqueName: \"kubernetes.io/projected/fdacd4fa-baa0-43a0-8945-a3cf42e7d990-kube-api-access-srrlr\") pod \"limitador-limitador-8588749977-cgmq7\" (UID: \"fdacd4fa-baa0-43a0-8945-a3cf42e7d990\") " pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" Sep 13 19:16:52.055313 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.055156 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fdacd4fa-baa0-43a0-8945-a3cf42e7d990-config-file\") pod \"limitador-limitador-8588749977-cgmq7\" (UID: \"fdacd4fa-baa0-43a0-8945-a3cf42e7d990\") " pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" Sep 13 19:16:52.156524 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.156467 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srrlr\" (UniqueName: \"kubernetes.io/projected/fdacd4fa-baa0-43a0-8945-a3cf42e7d990-kube-api-access-srrlr\") pod \"limitador-limitador-8588749977-cgmq7\" (UID: \"fdacd4fa-baa0-43a0-8945-a3cf42e7d990\") " pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" Sep 13 19:16:52.156710 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.156575 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fdacd4fa-baa0-43a0-8945-a3cf42e7d990-config-file\") pod \"limitador-limitador-8588749977-cgmq7\" (UID: \"fdacd4fa-baa0-43a0-8945-a3cf42e7d990\") " pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" Sep 13 19:16:52.157258 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.157220 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fdacd4fa-baa0-43a0-8945-a3cf42e7d990-config-file\") pod \"limitador-limitador-8588749977-cgmq7\" (UID: \"fdacd4fa-baa0-43a0-8945-a3cf42e7d990\") " pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" Sep 13 19:16:52.164198 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.164110 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srrlr\" (UniqueName: \"kubernetes.io/projected/fdacd4fa-baa0-43a0-8945-a3cf42e7d990-kube-api-access-srrlr\") pod \"limitador-limitador-8588749977-cgmq7\" (UID: \"fdacd4fa-baa0-43a0-8945-a3cf42e7d990\") " pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" Sep 13 19:16:52.228008 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.227964 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" Sep 13 19:16:52.503270 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.503221 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cgmq7"] Sep 13 19:16:52.505221 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:16:52.505195 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdacd4fa_baa0_43a0_8945_a3cf42e7d990.slice/crio-6a52f7a6d0855ffc721e7b5f0ea8ee5520428dc4dec00f0dec758fd2b4f4027a WatchSource:0}: Error finding container 6a52f7a6d0855ffc721e7b5f0ea8ee5520428dc4dec00f0dec758fd2b4f4027a: Status 404 returned error can't find the container with id 6a52f7a6d0855ffc721e7b5f0ea8ee5520428dc4dec00f0dec758fd2b4f4027a Sep 13 19:16:52.792766 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.792666 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-kt7z5"] Sep 13 19:16:52.795993 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.795965 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kt7z5" Sep 13 19:16:52.798423 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.798397 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-tt2l6\"" Sep 13 19:16:52.799817 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.799793 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kt7z5"] Sep 13 19:16:52.963797 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:52.963752 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j88hg\" (UniqueName: \"kubernetes.io/projected/cc1f3269-8987-4d14-88a8-1804695d007e-kube-api-access-j88hg\") pod \"authorino-59c974bd48-kt7z5\" (UID: \"cc1f3269-8987-4d14-88a8-1804695d007e\") " pod="kuadrant-system/authorino-59c974bd48-kt7z5" Sep 13 19:16:53.064689 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:53.064584 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j88hg\" (UniqueName: \"kubernetes.io/projected/cc1f3269-8987-4d14-88a8-1804695d007e-kube-api-access-j88hg\") pod \"authorino-59c974bd48-kt7z5\" (UID: \"cc1f3269-8987-4d14-88a8-1804695d007e\") " pod="kuadrant-system/authorino-59c974bd48-kt7z5" Sep 13 19:16:53.072152 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:53.072113 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j88hg\" (UniqueName: \"kubernetes.io/projected/cc1f3269-8987-4d14-88a8-1804695d007e-kube-api-access-j88hg\") pod \"authorino-59c974bd48-kt7z5\" (UID: \"cc1f3269-8987-4d14-88a8-1804695d007e\") " pod="kuadrant-system/authorino-59c974bd48-kt7z5" Sep 13 19:16:53.107156 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:53.107114 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kt7z5" Sep 13 19:16:53.375812 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:53.375723 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kt7z5"] Sep 13 19:16:53.379300 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:16:53.379267 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc1f3269_8987_4d14_88a8_1804695d007e.slice/crio-7b5594c4c91c4c0fada8d041bc02b61b88a9007b7bc83b5d9a388922aaa5bbea WatchSource:0}: Error finding container 7b5594c4c91c4c0fada8d041bc02b61b88a9007b7bc83b5d9a388922aaa5bbea: Status 404 returned error can't find the container with id 7b5594c4c91c4c0fada8d041bc02b61b88a9007b7bc83b5d9a388922aaa5bbea Sep 13 19:16:53.476866 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:53.476829 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kt7z5" event={"ID":"cc1f3269-8987-4d14-88a8-1804695d007e","Type":"ContainerStarted","Data":"7b5594c4c91c4c0fada8d041bc02b61b88a9007b7bc83b5d9a388922aaa5bbea"} Sep 13 19:16:53.478049 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:53.478013 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" event={"ID":"fdacd4fa-baa0-43a0-8945-a3cf42e7d990","Type":"ContainerStarted","Data":"6a52f7a6d0855ffc721e7b5f0ea8ee5520428dc4dec00f0dec758fd2b4f4027a"} Sep 13 19:16:54.484971 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:54.484934 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" event={"ID":"fdacd4fa-baa0-43a0-8945-a3cf42e7d990","Type":"ContainerStarted","Data":"45fea978046e3c61d472c1b479d5a42e33d38c9c62a4c8982151a4f0568cf86f"} Sep 13 19:16:54.485379 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:54.485001 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" Sep 13 19:16:54.500406 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:54.500346 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" podStartSLOduration=2.113273264 podStartE2EDuration="3.500327641s" totalImagesPullingTime="1.387054377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:16:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:16:54.4982997 +0000 UTC m=+765.879828730" watchObservedRunningTime="2026-09-13 19:16:54.500327641 +0000 UTC m=+765.881856669" Sep 13 19:16:57.502971 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:57.502934 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kt7z5" event={"ID":"cc1f3269-8987-4d14-88a8-1804695d007e","Type":"ContainerStarted","Data":"b1583f6eaf32415892a0c8c802550c6030e25f019c9728ce2725181ed0b53d9f"} Sep 13 19:16:57.521003 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:16:57.520943 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-kt7z5" podStartSLOduration=1.955205115 podStartE2EDuration="5.520923727s" totalImagesPullingTime="3.565718612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:16:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:16:57.517399172 +0000 UTC m=+768.898928211" watchObservedRunningTime="2026-09-13 19:16:57.520923727 +0000 UTC m=+768.902452755" Sep 13 19:17:05.491927 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:05.491893 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" Sep 13 19:17:06.090341 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:06.090296 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cgmq7"] Sep 13 19:17:06.090588 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:06.090538 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" podUID="fdacd4fa-baa0-43a0-8945-a3cf42e7d990" containerName="limitador" containerID="cri-o://45fea978046e3c61d472c1b479d5a42e33d38c9c62a4c8982151a4f0568cf86f" gracePeriod=30 Sep 13 19:17:06.553059 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:06.552981 2563 generic.go:356] "Generic (PLEG): container finished" podID="fdacd4fa-baa0-43a0-8945-a3cf42e7d990" containerID="45fea978046e3c61d472c1b479d5a42e33d38c9c62a4c8982151a4f0568cf86f" exitCode=0 Sep 13 19:17:06.553059 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:06.553026 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" event={"ID":"fdacd4fa-baa0-43a0-8945-a3cf42e7d990","Type":"ContainerDied","Data":"45fea978046e3c61d472c1b479d5a42e33d38c9c62a4c8982151a4f0568cf86f"} Sep 13 19:17:06.652086 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:06.652008 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" Sep 13 19:17:06.680061 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:06.680024 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srrlr\" (UniqueName: \"kubernetes.io/projected/fdacd4fa-baa0-43a0-8945-a3cf42e7d990-kube-api-access-srrlr\") pod \"fdacd4fa-baa0-43a0-8945-a3cf42e7d990\" (UID: \"fdacd4fa-baa0-43a0-8945-a3cf42e7d990\") " Sep 13 19:17:06.680285 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:06.680101 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fdacd4fa-baa0-43a0-8945-a3cf42e7d990-config-file\") pod \"fdacd4fa-baa0-43a0-8945-a3cf42e7d990\" (UID: \"fdacd4fa-baa0-43a0-8945-a3cf42e7d990\") " Sep 13 19:17:06.680498 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:06.680474 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdacd4fa-baa0-43a0-8945-a3cf42e7d990-config-file" (OuterVolumeSpecName: "config-file") pod "fdacd4fa-baa0-43a0-8945-a3cf42e7d990" (UID: "fdacd4fa-baa0-43a0-8945-a3cf42e7d990"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:17:06.682136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:06.682104 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdacd4fa-baa0-43a0-8945-a3cf42e7d990-kube-api-access-srrlr" (OuterVolumeSpecName: "kube-api-access-srrlr") pod "fdacd4fa-baa0-43a0-8945-a3cf42e7d990" (UID: "fdacd4fa-baa0-43a0-8945-a3cf42e7d990"). InnerVolumeSpecName "kube-api-access-srrlr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:17:06.781730 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:06.781696 2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fdacd4fa-baa0-43a0-8945-a3cf42e7d990-config-file\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:17:06.781730 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:06.781728 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srrlr\" (UniqueName: \"kubernetes.io/projected/fdacd4fa-baa0-43a0-8945-a3cf42e7d990-kube-api-access-srrlr\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:17:07.560201 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:07.560168 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" Sep 13 19:17:07.560201 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:07.560177 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cgmq7" event={"ID":"fdacd4fa-baa0-43a0-8945-a3cf42e7d990","Type":"ContainerDied","Data":"6a52f7a6d0855ffc721e7b5f0ea8ee5520428dc4dec00f0dec758fd2b4f4027a"} Sep 13 19:17:07.560756 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:07.560252 2563 scope.go:117] "RemoveContainer" containerID="45fea978046e3c61d472c1b479d5a42e33d38c9c62a4c8982151a4f0568cf86f" Sep 13 19:17:07.576591 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:07.576555 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cgmq7"] Sep 13 19:17:07.578364 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:07.578342 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cgmq7"] Sep 13 19:17:09.131567 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:09.131531 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdacd4fa-baa0-43a0-8945-a3cf42e7d990" path="/var/lib/kubelet/pods/fdacd4fa-baa0-43a0-8945-a3cf42e7d990/volumes" Sep 13 19:17:12.439515 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.439477 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-749q2"] Sep 13 19:17:12.440019 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.439924 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdacd4fa-baa0-43a0-8945-a3cf42e7d990" containerName="limitador" Sep 13 19:17:12.440019 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.439940 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdacd4fa-baa0-43a0-8945-a3cf42e7d990" containerName="limitador" Sep 13 19:17:12.440116 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.440036 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdacd4fa-baa0-43a0-8945-a3cf42e7d990" containerName="limitador" Sep 13 19:17:12.443559 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.443536 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" Sep 13 19:17:12.445941 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.445915 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-2f5nt\"" Sep 13 19:17:12.446093 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.445925 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 13 19:17:12.446278 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.446233 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 13 19:17:12.446888 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.446866 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 13 19:17:12.453660 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.453631 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-749q2"] Sep 13 19:17:12.536928 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.536884 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb5g6\" (UniqueName: \"kubernetes.io/projected/058f7c6c-0e74-4818-9780-10f0beb1db7d-kube-api-access-vb5g6\") pod \"postgres-69fb4594cf-749q2\" (UID: \"058f7c6c-0e74-4818-9780-10f0beb1db7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" Sep 13 19:17:12.537138 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.536965 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/058f7c6c-0e74-4818-9780-10f0beb1db7d-data\") pod \"postgres-69fb4594cf-749q2\" (UID: \"058f7c6c-0e74-4818-9780-10f0beb1db7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" Sep 13 19:17:12.637812 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.637772 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb5g6\" (UniqueName: \"kubernetes.io/projected/058f7c6c-0e74-4818-9780-10f0beb1db7d-kube-api-access-vb5g6\") pod \"postgres-69fb4594cf-749q2\" (UID: \"058f7c6c-0e74-4818-9780-10f0beb1db7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" Sep 13 19:17:12.638015 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.637829 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/058f7c6c-0e74-4818-9780-10f0beb1db7d-data\") pod \"postgres-69fb4594cf-749q2\" (UID: \"058f7c6c-0e74-4818-9780-10f0beb1db7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" Sep 13 19:17:12.638303 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.638280 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/058f7c6c-0e74-4818-9780-10f0beb1db7d-data\") pod \"postgres-69fb4594cf-749q2\" (UID: \"058f7c6c-0e74-4818-9780-10f0beb1db7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" Sep 13 19:17:12.645963 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.645935 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb5g6\" (UniqueName: \"kubernetes.io/projected/058f7c6c-0e74-4818-9780-10f0beb1db7d-kube-api-access-vb5g6\") pod \"postgres-69fb4594cf-749q2\" (UID: \"058f7c6c-0e74-4818-9780-10f0beb1db7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" Sep 13 19:17:12.755927 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:12.755885 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" Sep 13 19:17:13.027385 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:13.027356 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-749q2"] Sep 13 19:17:13.029440 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:17:13.029411 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod058f7c6c_0e74_4818_9780_10f0beb1db7d.slice/crio-7e5b15fa3755151313ad7315cb8ca3bad46fd3913847b810e3a45a8ac423efda WatchSource:0}: Error finding container 7e5b15fa3755151313ad7315cb8ca3bad46fd3913847b810e3a45a8ac423efda: Status 404 returned error can't find the container with id 7e5b15fa3755151313ad7315cb8ca3bad46fd3913847b810e3a45a8ac423efda Sep 13 19:17:13.595063 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:13.595027 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" event={"ID":"058f7c6c-0e74-4818-9780-10f0beb1db7d","Type":"ContainerStarted","Data":"7e5b15fa3755151313ad7315cb8ca3bad46fd3913847b810e3a45a8ac423efda"} Sep 13 19:17:16.614497 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:16.614459 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" event={"ID":"058f7c6c-0e74-4818-9780-10f0beb1db7d","Type":"ContainerStarted","Data":"bb78e575c386aacb88831a1bb00cb765f0be706451e79e64104e280d20f6f759"} Sep 13 19:17:16.614976 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:16.614578 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" Sep 13 19:17:16.629524 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:16.629464 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" podStartSLOduration=1.555412487 podStartE2EDuration="4.629446533s" totalImagesPullingTime="3.074034046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:17:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:17:16.627982459 +0000 UTC m=+788.009511485" watchObservedRunningTime="2026-09-13 19:17:16.629446533 +0000 UTC m=+788.010975561" Sep 13 19:17:22.649252 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:22.649202 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-749q2" Sep 13 19:17:25.490269 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:25.490212 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kt7z5"] Sep 13 19:17:25.490752 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:25.490479 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-kt7z5" podUID="cc1f3269-8987-4d14-88a8-1804695d007e" containerName="authorino" containerID="cri-o://b1583f6eaf32415892a0c8c802550c6030e25f019c9728ce2725181ed0b53d9f" gracePeriod=30 Sep 13 19:17:25.666046 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:25.665997 2563 generic.go:356] "Generic (PLEG): container finished" podID="cc1f3269-8987-4d14-88a8-1804695d007e" containerID="b1583f6eaf32415892a0c8c802550c6030e25f019c9728ce2725181ed0b53d9f" exitCode=0 Sep 13 19:17:25.666220 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:25.666086 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kt7z5" event={"ID":"cc1f3269-8987-4d14-88a8-1804695d007e","Type":"ContainerDied","Data":"b1583f6eaf32415892a0c8c802550c6030e25f019c9728ce2725181ed0b53d9f"} Sep 13 19:17:25.736556 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:25.736529 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kt7z5" Sep 13 19:17:25.859273 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:25.859167 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j88hg\" (UniqueName: \"kubernetes.io/projected/cc1f3269-8987-4d14-88a8-1804695d007e-kube-api-access-j88hg\") pod \"cc1f3269-8987-4d14-88a8-1804695d007e\" (UID: \"cc1f3269-8987-4d14-88a8-1804695d007e\") " Sep 13 19:17:25.861327 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:25.861291 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc1f3269-8987-4d14-88a8-1804695d007e-kube-api-access-j88hg" (OuterVolumeSpecName: "kube-api-access-j88hg") pod "cc1f3269-8987-4d14-88a8-1804695d007e" (UID: "cc1f3269-8987-4d14-88a8-1804695d007e"). InnerVolumeSpecName "kube-api-access-j88hg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:17:25.960656 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:25.960618 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j88hg\" (UniqueName: \"kubernetes.io/projected/cc1f3269-8987-4d14-88a8-1804695d007e-kube-api-access-j88hg\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:17:26.672512 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:26.672478 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kt7z5" Sep 13 19:17:26.672512 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:26.672491 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kt7z5" event={"ID":"cc1f3269-8987-4d14-88a8-1804695d007e","Type":"ContainerDied","Data":"7b5594c4c91c4c0fada8d041bc02b61b88a9007b7bc83b5d9a388922aaa5bbea"} Sep 13 19:17:26.673057 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:26.672545 2563 scope.go:117] "RemoveContainer" containerID="b1583f6eaf32415892a0c8c802550c6030e25f019c9728ce2725181ed0b53d9f" Sep 13 19:17:26.693537 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:26.693476 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kt7z5"] Sep 13 19:17:26.695331 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:26.695294 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kt7z5"] Sep 13 19:17:27.130968 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:27.130929 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc1f3269-8987-4d14-88a8-1804695d007e" path="/var/lib/kubelet/pods/cc1f3269-8987-4d14-88a8-1804695d007e/volumes" Sep 13 19:17:30.640761 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.640720 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-rqkt6"] Sep 13 19:17:30.641478 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.641452 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc1f3269-8987-4d14-88a8-1804695d007e" containerName="authorino" Sep 13 19:17:30.641478 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.641479 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc1f3269-8987-4d14-88a8-1804695d007e" containerName="authorino" Sep 13 19:17:30.641698 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.641604 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc1f3269-8987-4d14-88a8-1804695d007e" containerName="authorino" Sep 13 19:17:30.645357 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.645334 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:30.653613 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.647760 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 13 19:17:30.653613 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.647960 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 13 19:17:30.653613 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.648117 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 13 19:17:30.653613 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.648764 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9l67n\"" Sep 13 19:17:30.664072 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.663298 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-rqkt6"] Sep 13 19:17:30.698053 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.698015 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xldw\" (UniqueName: \"kubernetes.io/projected/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-kube-api-access-7xldw\") pod \"maas-controller-98f5d4d4f-rqkt6\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:30.698262 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.698145 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-webhook-cert\") pod \"maas-controller-98f5d4d4f-rqkt6\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:30.698262 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.698179 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-metrics-certs\") pod \"maas-controller-98f5d4d4f-rqkt6\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:30.799159 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.799112 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-webhook-cert\") pod \"maas-controller-98f5d4d4f-rqkt6\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:30.799384 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.799170 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-metrics-certs\") pod \"maas-controller-98f5d4d4f-rqkt6\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:30.799384 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.799215 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xldw\" (UniqueName: \"kubernetes.io/projected/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-kube-api-access-7xldw\") pod \"maas-controller-98f5d4d4f-rqkt6\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:30.799384 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:17:30.799309 2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 13 19:17:30.799384 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:17:30.799337 2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 13 19:17:30.799611 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:17:30.799407 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-webhook-cert podName:ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0 nodeName:}" failed. No retries permitted until 2026-09-13 19:17:31.299383733 +0000 UTC m=+802.680912755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-webhook-cert") pod "maas-controller-98f5d4d4f-rqkt6" (UID: "ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0") : secret "maas-controller-webhook-cert" not found Sep 13 19:17:30.799611 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:17:30.799426 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-metrics-certs podName:ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0 nodeName:}" failed. No retries permitted until 2026-09-13 19:17:31.29941759 +0000 UTC m=+802.680946602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-metrics-certs") pod "maas-controller-98f5d4d4f-rqkt6" (UID: "ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0") : secret "maas-controller-metrics-tls" not found Sep 13 19:17:30.808134 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.808097 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xldw\" (UniqueName: \"kubernetes.io/projected/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-kube-api-access-7xldw\") pod \"maas-controller-98f5d4d4f-rqkt6\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:30.830212 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.830170 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f7659468d-5gg4k"] Sep 13 19:17:30.833603 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.833578 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:30.841775 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.841737 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f7659468d-5gg4k"] Sep 13 19:17:30.900153 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.900051 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqn4k\" (UniqueName: \"kubernetes.io/projected/58de5cc7-1e42-4a41-9166-38acf02019d9-kube-api-access-pqn4k\") pod \"maas-controller-f7659468d-5gg4k\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:30.900314 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.900164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-metrics-certs\") pod \"maas-controller-f7659468d-5gg4k\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:30.900371 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:30.900315 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-webhook-cert\") pod \"maas-controller-f7659468d-5gg4k\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:31.001584 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.001488 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqn4k\" (UniqueName: \"kubernetes.io/projected/58de5cc7-1e42-4a41-9166-38acf02019d9-kube-api-access-pqn4k\") pod \"maas-controller-f7659468d-5gg4k\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:31.001750 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.001589 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-metrics-certs\") pod \"maas-controller-f7659468d-5gg4k\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:31.001750 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.001686 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-webhook-cert\") pod \"maas-controller-f7659468d-5gg4k\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:31.001750 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:17:31.001733 2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 13 19:17:31.001872 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:17:31.001802 2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 13 19:17:31.001872 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:17:31.001804 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-metrics-certs podName:58de5cc7-1e42-4a41-9166-38acf02019d9 nodeName:}" failed. No retries permitted until 2026-09-13 19:17:31.50178265 +0000 UTC m=+802.883311661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-metrics-certs") pod "maas-controller-f7659468d-5gg4k" (UID: "58de5cc7-1e42-4a41-9166-38acf02019d9") : secret "maas-controller-metrics-tls" not found Sep 13 19:17:31.001947 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:17:31.001898 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-webhook-cert podName:58de5cc7-1e42-4a41-9166-38acf02019d9 nodeName:}" failed. No retries permitted until 2026-09-13 19:17:31.501875775 +0000 UTC m=+802.883404801 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-webhook-cert") pod "maas-controller-f7659468d-5gg4k" (UID: "58de5cc7-1e42-4a41-9166-38acf02019d9") : secret "maas-controller-webhook-cert" not found Sep 13 19:17:31.010778 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.010744 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqn4k\" (UniqueName: \"kubernetes.io/projected/58de5cc7-1e42-4a41-9166-38acf02019d9-kube-api-access-pqn4k\") pod \"maas-controller-f7659468d-5gg4k\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:31.304782 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.304743 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-webhook-cert\") pod \"maas-controller-98f5d4d4f-rqkt6\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:31.305006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.304795 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-metrics-certs\") pod \"maas-controller-98f5d4d4f-rqkt6\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:31.307417 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.307388 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-webhook-cert\") pod \"maas-controller-98f5d4d4f-rqkt6\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:31.307532 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.307512 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-metrics-certs\") pod \"maas-controller-98f5d4d4f-rqkt6\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:31.505898 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.505854 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-webhook-cert\") pod \"maas-controller-f7659468d-5gg4k\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:31.506082 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.505948 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-metrics-certs\") pod \"maas-controller-f7659468d-5gg4k\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:31.508519 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.508490 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-webhook-cert\") pod \"maas-controller-f7659468d-5gg4k\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:31.508648 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.508547 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-metrics-certs\") pod \"maas-controller-f7659468d-5gg4k\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:31.560905 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.560809 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:31.746352 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.746312 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:31.833554 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:31.833466 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-rqkt6"] Sep 13 19:17:31.836810 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:17:31.836779 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddfe2db3_2e03_4f69_a9f9_fe3208d6a3c0.slice/crio-2473dbdb0e06bf275624ed5802c4b762569f8e2b648344dd8a79fd6730f5e869 WatchSource:0}: Error finding container 2473dbdb0e06bf275624ed5802c4b762569f8e2b648344dd8a79fd6730f5e869: Status 404 returned error can't find the container with id 2473dbdb0e06bf275624ed5802c4b762569f8e2b648344dd8a79fd6730f5e869 Sep 13 19:17:32.018898 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:32.018830 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f7659468d-5gg4k"] Sep 13 19:17:32.021205 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:17:32.021180 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58de5cc7_1e42_4a41_9166_38acf02019d9.slice/crio-a7355747f329ef086a8d9424cbf753e6b6eb8d47efb53d8855541d251430cac1 WatchSource:0}: Error finding container a7355747f329ef086a8d9424cbf753e6b6eb8d47efb53d8855541d251430cac1: Status 404 returned error can't find the container with id a7355747f329ef086a8d9424cbf753e6b6eb8d47efb53d8855541d251430cac1 Sep 13 19:17:32.710940 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:32.710893 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" event={"ID":"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0","Type":"ContainerStarted","Data":"2473dbdb0e06bf275624ed5802c4b762569f8e2b648344dd8a79fd6730f5e869"} Sep 13 19:17:32.712495 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:32.712415 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f7659468d-5gg4k" event={"ID":"58de5cc7-1e42-4a41-9166-38acf02019d9","Type":"ContainerStarted","Data":"a7355747f329ef086a8d9424cbf753e6b6eb8d47efb53d8855541d251430cac1"} Sep 13 19:17:34.726086 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:34.726042 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" event={"ID":"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0","Type":"ContainerStarted","Data":"439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173"} Sep 13 19:17:34.726531 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:34.726119 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:34.727634 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:34.727608 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f7659468d-5gg4k" event={"ID":"58de5cc7-1e42-4a41-9166-38acf02019d9","Type":"ContainerStarted","Data":"55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15"} Sep 13 19:17:34.727733 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:34.727659 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:34.746331 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:34.746206 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" podStartSLOduration=2.084993247 podStartE2EDuration="4.746188024s" totalImagesPullingTime="2.661194777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:17:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:17:34.744793438 +0000 UTC m=+806.126322468" watchObservedRunningTime="2026-09-13 19:17:34.746188024 +0000 UTC m=+806.127717053" Sep 13 19:17:34.763809 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:34.763718 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f7659468d-5gg4k" podStartSLOduration=2.298692876 podStartE2EDuration="4.763684024s" totalImagesPullingTime="2.464991148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:17:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:17:34.761416812 +0000 UTC m=+806.142945855" watchObservedRunningTime="2026-09-13 19:17:34.763684024 +0000 UTC m=+806.145213068" Sep 13 19:17:45.736884 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:45.736851 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:17:45.737451 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:45.736926 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:45.797010 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:45.796967 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-rqkt6"] Sep 13 19:17:45.797278 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:45.797206 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" podUID="ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0" containerName="manager" containerID="cri-o://439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173" gracePeriod=30 Sep 13 19:17:46.053645 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.053615 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:46.154140 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.154105 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xldw\" (UniqueName: \"kubernetes.io/projected/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-kube-api-access-7xldw\") pod \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " Sep 13 19:17:46.154333 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.154207 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-metrics-certs\") pod \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " Sep 13 19:17:46.154333 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.154267 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-webhook-cert\") pod \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\" (UID: \"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0\") " Sep 13 19:17:46.156599 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.156567 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0" (UID: "ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:17:46.156757 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.156624 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-kube-api-access-7xldw" (OuterVolumeSpecName: "kube-api-access-7xldw") pod "ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0" (UID: "ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0"). InnerVolumeSpecName "kube-api-access-7xldw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:17:46.156757 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.156638 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0" (UID: "ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:17:46.256111 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.256052 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xldw\" (UniqueName: \"kubernetes.io/projected/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-kube-api-access-7xldw\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:17:46.256111 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.256106 2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-metrics-certs\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:17:46.256111 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.256120 2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0-webhook-cert\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:17:46.803851 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.803810 2563 generic.go:356] "Generic (PLEG): container finished" podID="ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0" containerID="439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173" exitCode=0 Sep 13 19:17:46.804297 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.803865 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" Sep 13 19:17:46.804297 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.803893 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" event={"ID":"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0","Type":"ContainerDied","Data":"439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173"} Sep 13 19:17:46.804297 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.803965 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98f5d4d4f-rqkt6" event={"ID":"ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0","Type":"ContainerDied","Data":"2473dbdb0e06bf275624ed5802c4b762569f8e2b648344dd8a79fd6730f5e869"} Sep 13 19:17:46.804297 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.803988 2563 scope.go:117] "RemoveContainer" containerID="439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173" Sep 13 19:17:46.813768 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.813745 2563 scope.go:117] "RemoveContainer" containerID="439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173" Sep 13 19:17:46.814106 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:17:46.814085 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173\": container with ID starting with 439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173 not found: ID does not exist" containerID="439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173" Sep 13 19:17:46.814151 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.814116 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173"} err="failed to get container status \"439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173\": rpc error: code = NotFound desc = could not find container \"439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173\": container with ID starting with 439cacea04c9149d012c447b7d513e79f558a9254923c6f8587b60602ff85173 not found: ID does not exist" Sep 13 19:17:46.825180 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.825147 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-rqkt6"] Sep 13 19:17:46.827056 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:46.827027 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-rqkt6"] Sep 13 19:17:47.131030 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:17:47.130948 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0" path="/var/lib/kubelet/pods/ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0/volumes" Sep 13 19:18:07.969770 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:07.969729 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx"] Sep 13 19:18:07.970337 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:07.970319 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0" containerName="manager" Sep 13 19:18:07.970389 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:07.970340 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0" containerName="manager" Sep 13 19:18:07.970455 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:07.970444 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddfe2db3-2e03-4f69-a9f9-fe3208d6a3c0" containerName="manager" Sep 13 19:18:07.977007 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:07.976978 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:07.979692 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:07.979664 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hkj5t\"" Sep 13 19:18:07.980149 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:07.980125 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 13 19:18:07.980351 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:07.980336 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 13 19:18:07.980948 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:07.980921 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx"] Sep 13 19:18:08.047731 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.047679 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2p9g\" (UniqueName: \"kubernetes.io/projected/e999e2af-5cfd-4d79-a511-a90bca848e5b-kube-api-access-r2p9g\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.047923 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.047740 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e999e2af-5cfd-4d79-a511-a90bca848e5b-maas-api-tls\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.047923 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.047769 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e999e2af-5cfd-4d79-a511-a90bca848e5b-openshift-service-ca\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.047923 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.047903 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e999e2af-5cfd-4d79-a511-a90bca848e5b-metrics-certs\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.148589 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.148546 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e999e2af-5cfd-4d79-a511-a90bca848e5b-metrics-certs\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.148783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.148619 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2p9g\" (UniqueName: \"kubernetes.io/projected/e999e2af-5cfd-4d79-a511-a90bca848e5b-kube-api-access-r2p9g\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.148783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.148660 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e999e2af-5cfd-4d79-a511-a90bca848e5b-maas-api-tls\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.148783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.148680 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e999e2af-5cfd-4d79-a511-a90bca848e5b-openshift-service-ca\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.149079 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:18:08.149054 2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found Sep 13 19:18:08.149345 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:18:08.149057 2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Sep 13 19:18:08.149345 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:18:08.149265 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e999e2af-5cfd-4d79-a511-a90bca848e5b-metrics-certs podName:e999e2af-5cfd-4d79-a511-a90bca848e5b nodeName:}" failed. No retries permitted until 2026-09-13 19:18:08.64922387 +0000 UTC m=+840.030752887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e999e2af-5cfd-4d79-a511-a90bca848e5b-metrics-certs") pod "maas-api-59f6566bbc-xtdmx" (UID: "e999e2af-5cfd-4d79-a511-a90bca848e5b") : secret "maas-api-metrics-tls" not found Sep 13 19:18:08.149345 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:18:08.149318 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e999e2af-5cfd-4d79-a511-a90bca848e5b-maas-api-tls podName:e999e2af-5cfd-4d79-a511-a90bca848e5b nodeName:}" failed. No retries permitted until 2026-09-13 19:18:08.649297472 +0000 UTC m=+840.030826483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e999e2af-5cfd-4d79-a511-a90bca848e5b-maas-api-tls") pod "maas-api-59f6566bbc-xtdmx" (UID: "e999e2af-5cfd-4d79-a511-a90bca848e5b") : secret "maas-api-serving-cert" not found Sep 13 19:18:08.149928 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.149905 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e999e2af-5cfd-4d79-a511-a90bca848e5b-openshift-service-ca\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.159624 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.159591 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2p9g\" (UniqueName: \"kubernetes.io/projected/e999e2af-5cfd-4d79-a511-a90bca848e5b-kube-api-access-r2p9g\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.395675 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.395576 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl"] Sep 13 19:18:08.399435 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.399405 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" Sep 13 19:18:08.403623 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.403594 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ztnfv\"" Sep 13 19:18:08.403795 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.403671 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 13 19:18:08.415609 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.415571 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl"] Sep 13 19:18:08.437078 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.437037 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-h84nw"] Sep 13 19:18:08.440819 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.440787 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" Sep 13 19:18:08.452554 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.452515 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbxbd\" (UniqueName: \"kubernetes.io/projected/f64a3896-9044-479f-9500-65802339e7a6-kube-api-access-jbxbd\") pod \"payload-pre-processing-59fbc47bf8-4cgwl\" (UID: \"f64a3896-9044-479f-9500-65802339e7a6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" Sep 13 19:18:08.452720 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.452612 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f64a3896-9044-479f-9500-65802339e7a6-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-4cgwl\" (UID: \"f64a3896-9044-479f-9500-65802339e7a6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" Sep 13 19:18:08.452906 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.452880 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-h84nw"] Sep 13 19:18:08.553518 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.553477 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbxbd\" (UniqueName: \"kubernetes.io/projected/f64a3896-9044-479f-9500-65802339e7a6-kube-api-access-jbxbd\") pod \"payload-pre-processing-59fbc47bf8-4cgwl\" (UID: \"f64a3896-9044-479f-9500-65802339e7a6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" Sep 13 19:18:08.553708 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.553544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a347145-36da-4d00-8983-e0bb69801299-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-h84nw\" (UID: \"1a347145-36da-4d00-8983-e0bb69801299\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" Sep 13 19:18:08.553708 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.553579 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f64a3896-9044-479f-9500-65802339e7a6-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-4cgwl\" (UID: \"f64a3896-9044-479f-9500-65802339e7a6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" Sep 13 19:18:08.553708 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.553647 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjrwg\" (UniqueName: \"kubernetes.io/projected/1a347145-36da-4d00-8983-e0bb69801299-kube-api-access-zjrwg\") pod \"payload-processing-5b54b7fbb7-h84nw\" (UID: \"1a347145-36da-4d00-8983-e0bb69801299\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" Sep 13 19:18:08.554415 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.554389 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f64a3896-9044-479f-9500-65802339e7a6-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-4cgwl\" (UID: \"f64a3896-9044-479f-9500-65802339e7a6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" Sep 13 19:18:08.561396 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.561363 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbxbd\" (UniqueName: \"kubernetes.io/projected/f64a3896-9044-479f-9500-65802339e7a6-kube-api-access-jbxbd\") pod \"payload-pre-processing-59fbc47bf8-4cgwl\" (UID: \"f64a3896-9044-479f-9500-65802339e7a6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" Sep 13 19:18:08.655007 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.654907 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e999e2af-5cfd-4d79-a511-a90bca848e5b-maas-api-tls\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.655007 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.654990 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a347145-36da-4d00-8983-e0bb69801299-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-h84nw\" (UID: \"1a347145-36da-4d00-8983-e0bb69801299\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" Sep 13 19:18:08.655261 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.655033 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e999e2af-5cfd-4d79-a511-a90bca848e5b-metrics-certs\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.655261 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.655059 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjrwg\" (UniqueName: \"kubernetes.io/projected/1a347145-36da-4d00-8983-e0bb69801299-kube-api-access-zjrwg\") pod \"payload-processing-5b54b7fbb7-h84nw\" (UID: \"1a347145-36da-4d00-8983-e0bb69801299\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" Sep 13 19:18:08.655953 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.655923 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a347145-36da-4d00-8983-e0bb69801299-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-h84nw\" (UID: \"1a347145-36da-4d00-8983-e0bb69801299\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" Sep 13 19:18:08.657811 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.657787 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e999e2af-5cfd-4d79-a511-a90bca848e5b-maas-api-tls\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.657934 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.657840 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e999e2af-5cfd-4d79-a511-a90bca848e5b-metrics-certs\") pod \"maas-api-59f6566bbc-xtdmx\" (UID: \"e999e2af-5cfd-4d79-a511-a90bca848e5b\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.662731 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.662702 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjrwg\" (UniqueName: \"kubernetes.io/projected/1a347145-36da-4d00-8983-e0bb69801299-kube-api-access-zjrwg\") pod \"payload-processing-5b54b7fbb7-h84nw\" (UID: \"1a347145-36da-4d00-8983-e0bb69801299\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" Sep 13 19:18:08.718315 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.718280 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" Sep 13 19:18:08.754491 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.754453 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" Sep 13 19:18:08.892502 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.892456 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:08.997700 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:08.997673 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl"] Sep 13 19:18:09.000758 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:18:09.000727 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64a3896_9044_479f_9500_65802339e7a6.slice/crio-27363dad248b44ca781b86af675c0b83df34922cebac1340bb8ea3a9aca360a4 WatchSource:0}: Error finding container 27363dad248b44ca781b86af675c0b83df34922cebac1340bb8ea3a9aca360a4: Status 404 returned error can't find the container with id 27363dad248b44ca781b86af675c0b83df34922cebac1340bb8ea3a9aca360a4 Sep 13 19:18:09.039030 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:09.038999 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-h84nw"] Sep 13 19:18:09.042947 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:18:09.042918 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a347145_36da_4d00_8983_e0bb69801299.slice/crio-aead2d290a767e53e6e9457214c854bd3aa990fe4885e974784c49f1cf854302 WatchSource:0}: Error finding container aead2d290a767e53e6e9457214c854bd3aa990fe4885e974784c49f1cf854302: Status 404 returned error can't find the container with id aead2d290a767e53e6e9457214c854bd3aa990fe4885e974784c49f1cf854302 Sep 13 19:18:09.182880 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:09.182716 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx"] Sep 13 19:18:09.185164 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:18:09.185132 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode999e2af_5cfd_4d79_a511_a90bca848e5b.slice/crio-6e5c0c0f4537786303601aecd9bcc5ea5bdd859b8255288fe821f453c0fd1f9d WatchSource:0}: Error finding container 6e5c0c0f4537786303601aecd9bcc5ea5bdd859b8255288fe821f453c0fd1f9d: Status 404 returned error can't find the container with id 6e5c0c0f4537786303601aecd9bcc5ea5bdd859b8255288fe821f453c0fd1f9d Sep 13 19:18:09.931492 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:09.931447 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" event={"ID":"e999e2af-5cfd-4d79-a511-a90bca848e5b","Type":"ContainerStarted","Data":"6e5c0c0f4537786303601aecd9bcc5ea5bdd859b8255288fe821f453c0fd1f9d"} Sep 13 19:18:09.933530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:09.933492 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" event={"ID":"f64a3896-9044-479f-9500-65802339e7a6","Type":"ContainerStarted","Data":"27363dad248b44ca781b86af675c0b83df34922cebac1340bb8ea3a9aca360a4"} Sep 13 19:18:09.935098 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:09.935065 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" event={"ID":"1a347145-36da-4d00-8983-e0bb69801299","Type":"ContainerStarted","Data":"aead2d290a767e53e6e9457214c854bd3aa990fe4885e974784c49f1cf854302"} Sep 13 19:18:11.951524 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:11.951472 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" event={"ID":"1a347145-36da-4d00-8983-e0bb69801299","Type":"ContainerStarted","Data":"4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2"} Sep 13 19:18:11.952526 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:11.952497 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" Sep 13 19:18:11.957407 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:11.957363 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" event={"ID":"e999e2af-5cfd-4d79-a511-a90bca848e5b","Type":"ContainerStarted","Data":"f1253480d1997ba22826322fca7cd136b88626e09586f012bc47fd1f714c6a7b"} Sep 13 19:18:11.958037 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:11.958010 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:11.960336 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:11.960299 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" event={"ID":"f64a3896-9044-479f-9500-65802339e7a6","Type":"ContainerStarted","Data":"3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703"} Sep 13 19:18:11.960968 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:11.960939 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" Sep 13 19:18:11.969258 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:11.969130 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" podStartSLOduration=1.684030866 podStartE2EDuration="3.969106436s" totalImagesPullingTime="2.28507557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:18:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:18:11.96677429 +0000 UTC m=+843.348303320" watchObservedRunningTime="2026-09-13 19:18:11.969106436 +0000 UTC m=+843.350635468" Sep 13 19:18:11.985367 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:11.985301 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" podStartSLOduration=2.8402374310000003 podStartE2EDuration="4.985276027s" totalImagesPullingTime="2.145038596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:18:11.983427945 +0000 UTC m=+843.364956977" watchObservedRunningTime="2026-09-13 19:18:11.985276027 +0000 UTC m=+843.366805058" Sep 13 19:18:11.998476 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:11.998411 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" podStartSLOduration=1.670770949 podStartE2EDuration="3.998388789s" totalImagesPullingTime="2.32761784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:18:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:18:11.997057527 +0000 UTC m=+843.378586558" watchObservedRunningTime="2026-09-13 19:18:11.998388789 +0000 UTC m=+843.379917818" Sep 13 19:18:17.999105 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:17.999074 2563 generic.go:356] "Generic (PLEG): container finished" podID="ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb" containerID="821f8918b1be906936c02a062d672df90757eda3ec5d6e69a7ca23d3e9adc6a1" exitCode=0 Sep 13 19:18:17.999548 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:17.999147 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" event={"ID":"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb","Type":"ContainerDied","Data":"821f8918b1be906936c02a062d672df90757eda3ec5d6e69a7ca23d3e9adc6a1"} Sep 13 19:18:17.999646 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:17.999629 2563 scope.go:117] "RemoveContainer" containerID="821f8918b1be906936c02a062d672df90757eda3ec5d6e69a7ca23d3e9adc6a1" Sep 13 19:18:18.000749 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:18.000715 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 13 19:18:18.000865 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:18.000798 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 13 19:18:18.000865 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:18.000835 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 13 19:18:18.980941 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:18.980911 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-xtdmx" Sep 13 19:18:19.009090 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:19.009056 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" event={"ID":"ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb","Type":"ContainerStarted","Data":"a54c7aff02e04ef53b1176c8800f2d039debc03d97f64d02f2ecbea758e80899"} Sep 13 19:18:19.086699 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:19.086664 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:18:19.086929 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:19.086905 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-b9dvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Sep 13 19:18:19.086980 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:19.086958 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" podUID="ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Sep 13 19:18:20.086807 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:20.086771 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-b9dvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Sep 13 19:18:20.087204 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:20.086829 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" podUID="ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Sep 13 19:18:21.086136 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:21.086100 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-b9dvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Sep 13 19:18:21.086349 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:21.086174 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" podUID="ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Sep 13 19:18:22.086868 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:22.086832 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-b9dvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Sep 13 19:18:22.087293 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:22.086894 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" podUID="ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Sep 13 19:18:23.087081 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:23.087042 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-b9dvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Sep 13 19:18:23.087524 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:23.087099 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" podUID="ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Sep 13 19:18:23.977065 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:23.977035 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" Sep 13 19:18:23.977274 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:23.977151 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" Sep 13 19:18:24.086296 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:24.086225 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-b9dvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Sep 13 19:18:24.086468 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:24.086320 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" podUID="ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Sep 13 19:18:25.090187 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:25.090151 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:18:25.090580 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:25.090457 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:18:25.091130 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:18:25.091113 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b9dvd" Sep 13 19:19:09.088016 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:19:09.087924 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/1.log" Sep 13 19:19:09.100672 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:19:09.100645 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/1.log" Sep 13 19:21:09.145563 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:09.145522 2563 scope.go:117] "RemoveContainer" containerID="c3f1f6dab95d4ea9c45796f3c8ec4e429512a0b3438f776e8ef92ea8e4f3fe4e" Sep 13 19:21:23.342288 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.342216 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c5c57c68f-tbxgn"] Sep 13 19:21:23.347454 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.347422 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c5c57c68f-tbxgn" Sep 13 19:21:23.349925 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.349892 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 13 19:21:23.350330 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.350304 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/871de01b-61f4-4b1e-a2a3-771b6f33af1e-tls-cert\") pod \"authorino-7c5c57c68f-tbxgn\" (UID: \"871de01b-61f4-4b1e-a2a3-771b6f33af1e\") " pod="kuadrant-system/authorino-7c5c57c68f-tbxgn" Sep 13 19:21:23.350435 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.350366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z26jd\" (UniqueName: \"kubernetes.io/projected/871de01b-61f4-4b1e-a2a3-771b6f33af1e-kube-api-access-z26jd\") pod \"authorino-7c5c57c68f-tbxgn\" (UID: \"871de01b-61f4-4b1e-a2a3-771b6f33af1e\") " pod="kuadrant-system/authorino-7c5c57c68f-tbxgn" Sep 13 19:21:23.351360 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.351339 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-tt2l6\"" Sep 13 19:21:23.359587 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.359555 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c5c57c68f-tbxgn"] Sep 13 19:21:23.451516 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.451477 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z26jd\" (UniqueName: \"kubernetes.io/projected/871de01b-61f4-4b1e-a2a3-771b6f33af1e-kube-api-access-z26jd\") pod \"authorino-7c5c57c68f-tbxgn\" (UID: \"871de01b-61f4-4b1e-a2a3-771b6f33af1e\") " pod="kuadrant-system/authorino-7c5c57c68f-tbxgn" Sep 13 19:21:23.451701 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.451612 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/871de01b-61f4-4b1e-a2a3-771b6f33af1e-tls-cert\") pod \"authorino-7c5c57c68f-tbxgn\" (UID: \"871de01b-61f4-4b1e-a2a3-771b6f33af1e\") " pod="kuadrant-system/authorino-7c5c57c68f-tbxgn" Sep 13 19:21:23.454252 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.454201 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/871de01b-61f4-4b1e-a2a3-771b6f33af1e-tls-cert\") pod \"authorino-7c5c57c68f-tbxgn\" (UID: \"871de01b-61f4-4b1e-a2a3-771b6f33af1e\") " pod="kuadrant-system/authorino-7c5c57c68f-tbxgn" Sep 13 19:21:23.459053 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.459024 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z26jd\" (UniqueName: \"kubernetes.io/projected/871de01b-61f4-4b1e-a2a3-771b6f33af1e-kube-api-access-z26jd\") pod \"authorino-7c5c57c68f-tbxgn\" (UID: \"871de01b-61f4-4b1e-a2a3-771b6f33af1e\") " pod="kuadrant-system/authorino-7c5c57c68f-tbxgn" Sep 13 19:21:23.658413 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.658314 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c5c57c68f-tbxgn" Sep 13 19:21:23.931195 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.931167 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c5c57c68f-tbxgn"] Sep 13 19:21:23.933653 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:21:23.933623 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod871de01b_61f4_4b1e_a2a3_771b6f33af1e.slice/crio-0e36c9ab495b4f8a88d1ccb727b629c3b64e79867ba010bfa1e609135c0d8445 WatchSource:0}: Error finding container 0e36c9ab495b4f8a88d1ccb727b629c3b64e79867ba010bfa1e609135c0d8445: Status 404 returned error can't find the container with id 0e36c9ab495b4f8a88d1ccb727b629c3b64e79867ba010bfa1e609135c0d8445 Sep 13 19:21:23.935063 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:23.935042 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 13 19:21:24.021347 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:24.021306 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c5c57c68f-tbxgn" event={"ID":"871de01b-61f4-4b1e-a2a3-771b6f33af1e","Type":"ContainerStarted","Data":"0e36c9ab495b4f8a88d1ccb727b629c3b64e79867ba010bfa1e609135c0d8445"} Sep 13 19:21:25.029482 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.029442 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c5c57c68f-tbxgn" event={"ID":"871de01b-61f4-4b1e-a2a3-771b6f33af1e","Type":"ContainerStarted","Data":"931dbf7a61fe5c097502e4d42ce75f7ee98a98cc6e64b501eb3eaadde2d03f89"} Sep 13 19:21:25.045157 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.045096 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7c5c57c68f-tbxgn" podStartSLOduration=1.3965605669999999 podStartE2EDuration="2.045076288s" totalImagesPullingTime="648.515721ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:21:25.043581356 +0000 UTC m=+1036.425110386" watchObservedRunningTime="2026-09-13 19:21:25.045076288 +0000 UTC m=+1036.426605317" Sep 13 19:21:25.384324 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.383935 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f7659468d-5gg4k"] Sep 13 19:21:25.384324 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.384166 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-f7659468d-5gg4k" podUID="58de5cc7-1e42-4a41-9166-38acf02019d9" containerName="manager" containerID="cri-o://55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15" gracePeriod=30 Sep 13 19:21:25.606967 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.606912 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-h84nw"] Sep 13 19:21:25.607210 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.607176 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" podUID="1a347145-36da-4d00-8983-e0bb69801299" containerName="payload-processing" containerID="cri-o://4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2" gracePeriod=30 Sep 13 19:21:25.737056 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.737019 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl"] Sep 13 19:21:25.737921 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.737275 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" podUID="f64a3896-9044-479f-9500-65802339e7a6" containerName="payload-pre-processing" containerID="cri-o://3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703" gracePeriod=30 Sep 13 19:21:25.748285 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.748105 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:21:25.770117 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.770058 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqn4k\" (UniqueName: \"kubernetes.io/projected/58de5cc7-1e42-4a41-9166-38acf02019d9-kube-api-access-pqn4k\") pod \"58de5cc7-1e42-4a41-9166-38acf02019d9\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " Sep 13 19:21:25.770339 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.770182 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-webhook-cert\") pod \"58de5cc7-1e42-4a41-9166-38acf02019d9\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " Sep 13 19:21:25.770339 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.770284 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-metrics-certs\") pod \"58de5cc7-1e42-4a41-9166-38acf02019d9\" (UID: \"58de5cc7-1e42-4a41-9166-38acf02019d9\") " Sep 13 19:21:25.773051 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.773005 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "58de5cc7-1e42-4a41-9166-38acf02019d9" (UID: "58de5cc7-1e42-4a41-9166-38acf02019d9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:21:25.773051 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.773010 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "58de5cc7-1e42-4a41-9166-38acf02019d9" (UID: "58de5cc7-1e42-4a41-9166-38acf02019d9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 19:21:25.773337 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.773133 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58de5cc7-1e42-4a41-9166-38acf02019d9-kube-api-access-pqn4k" (OuterVolumeSpecName: "kube-api-access-pqn4k") pod "58de5cc7-1e42-4a41-9166-38acf02019d9" (UID: "58de5cc7-1e42-4a41-9166-38acf02019d9"). InnerVolumeSpecName "kube-api-access-pqn4k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:21:25.874930 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.874845 2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-webhook-cert\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:21:25.874930 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.874884 2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58de5cc7-1e42-4a41-9166-38acf02019d9-metrics-certs\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:21:25.874930 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.874902 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqn4k\" (UniqueName: \"kubernetes.io/projected/58de5cc7-1e42-4a41-9166-38acf02019d9-kube-api-access-pqn4k\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:21:25.912754 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.912726 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" Sep 13 19:21:25.975854 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.975801 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjrwg\" (UniqueName: \"kubernetes.io/projected/1a347145-36da-4d00-8983-e0bb69801299-kube-api-access-zjrwg\") pod \"1a347145-36da-4d00-8983-e0bb69801299\" (UID: \"1a347145-36da-4d00-8983-e0bb69801299\") " Sep 13 19:21:25.976152 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.975950 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a347145-36da-4d00-8983-e0bb69801299-plugins-config-volume\") pod \"1a347145-36da-4d00-8983-e0bb69801299\" (UID: \"1a347145-36da-4d00-8983-e0bb69801299\") " Sep 13 19:21:25.976751 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.976701 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a347145-36da-4d00-8983-e0bb69801299-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a347145-36da-4d00-8983-e0bb69801299" (UID: "1a347145-36da-4d00-8983-e0bb69801299"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:21:25.979503 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:25.979467 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a347145-36da-4d00-8983-e0bb69801299-kube-api-access-zjrwg" (OuterVolumeSpecName: "kube-api-access-zjrwg") pod "1a347145-36da-4d00-8983-e0bb69801299" (UID: "1a347145-36da-4d00-8983-e0bb69801299"). InnerVolumeSpecName "kube-api-access-zjrwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:21:26.035793 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.035765 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" Sep 13 19:21:26.036708 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.036681 2563 generic.go:356] "Generic (PLEG): container finished" podID="f64a3896-9044-479f-9500-65802339e7a6" containerID="3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703" exitCode=0 Sep 13 19:21:26.036841 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.036759 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" event={"ID":"f64a3896-9044-479f-9500-65802339e7a6","Type":"ContainerDied","Data":"3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703"} Sep 13 19:21:26.036906 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.036868 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" event={"ID":"f64a3896-9044-479f-9500-65802339e7a6","Type":"ContainerDied","Data":"27363dad248b44ca781b86af675c0b83df34922cebac1340bb8ea3a9aca360a4"} Sep 13 19:21:26.036952 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.036902 2563 scope.go:117] "RemoveContainer" containerID="3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703" Sep 13 19:21:26.038209 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.038185 2563 generic.go:356] "Generic (PLEG): container finished" podID="58de5cc7-1e42-4a41-9166-38acf02019d9" containerID="55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15" exitCode=0 Sep 13 19:21:26.038337 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.038270 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f7659468d-5gg4k" Sep 13 19:21:26.038337 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.038269 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f7659468d-5gg4k" event={"ID":"58de5cc7-1e42-4a41-9166-38acf02019d9","Type":"ContainerDied","Data":"55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15"} Sep 13 19:21:26.038337 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.038326 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f7659468d-5gg4k" event={"ID":"58de5cc7-1e42-4a41-9166-38acf02019d9","Type":"ContainerDied","Data":"a7355747f329ef086a8d9424cbf753e6b6eb8d47efb53d8855541d251430cac1"} Sep 13 19:21:26.040216 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.040188 2563 generic.go:356] "Generic (PLEG): container finished" podID="1a347145-36da-4d00-8983-e0bb69801299" containerID="4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2" exitCode=0 Sep 13 19:21:26.040363 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.040286 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" Sep 13 19:21:26.040426 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.040279 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" event={"ID":"1a347145-36da-4d00-8983-e0bb69801299","Type":"ContainerDied","Data":"4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2"} Sep 13 19:21:26.040426 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.040412 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-h84nw" event={"ID":"1a347145-36da-4d00-8983-e0bb69801299","Type":"ContainerDied","Data":"aead2d290a767e53e6e9457214c854bd3aa990fe4885e974784c49f1cf854302"} Sep 13 19:21:26.047398 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.047374 2563 scope.go:117] "RemoveContainer" containerID="3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703" Sep 13 19:21:26.047858 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:21:26.047832 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703\": container with ID starting with 3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703 not found: ID does not exist" containerID="3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703" Sep 13 19:21:26.047964 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.047874 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703"} err="failed to get container status \"3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703\": rpc error: code = NotFound desc = could not find container \"3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703\": container with ID starting with 3a87a407fe3bac3556e872d0f6a7c704ec728843869d279ad424fb65e8e91703 not found: ID does not exist" Sep 13 19:21:26.047964 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.047911 2563 scope.go:117] "RemoveContainer" containerID="55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15" Sep 13 19:21:26.062077 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.062048 2563 scope.go:117] "RemoveContainer" containerID="55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15" Sep 13 19:21:26.063625 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:21:26.063596 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15\": container with ID starting with 55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15 not found: ID does not exist" containerID="55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15" Sep 13 19:21:26.063781 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.063636 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15"} err="failed to get container status \"55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15\": rpc error: code = NotFound desc = could not find container \"55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15\": container with ID starting with 55d4e1be0125bc40a0e492d974cec9cd41ba673d7f83c81bf0adadad57669e15 not found: ID does not exist" Sep 13 19:21:26.063781 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.063664 2563 scope.go:117] "RemoveContainer" containerID="4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2" Sep 13 19:21:26.072942 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.072907 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f7659468d-5gg4k"] Sep 13 19:21:26.076322 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.076275 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f7659468d-5gg4k"] Sep 13 19:21:26.076737 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.076720 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbxbd\" (UniqueName: \"kubernetes.io/projected/f64a3896-9044-479f-9500-65802339e7a6-kube-api-access-jbxbd\") pod \"f64a3896-9044-479f-9500-65802339e7a6\" (UID: \"f64a3896-9044-479f-9500-65802339e7a6\") " Sep 13 19:21:26.076796 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.076758 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f64a3896-9044-479f-9500-65802339e7a6-plugins-config-volume\") pod \"f64a3896-9044-479f-9500-65802339e7a6\" (UID: \"f64a3896-9044-479f-9500-65802339e7a6\") " Sep 13 19:21:26.076940 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.076894 2563 scope.go:117] "RemoveContainer" containerID="4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2" Sep 13 19:21:26.077228 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.077195 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a347145-36da-4d00-8983-e0bb69801299-plugins-config-volume\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:21:26.077470 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.077447 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjrwg\" (UniqueName: \"kubernetes.io/projected/1a347145-36da-4d00-8983-e0bb69801299-kube-api-access-zjrwg\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:21:26.077599 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.077558 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f64a3896-9044-479f-9500-65802339e7a6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f64a3896-9044-479f-9500-65802339e7a6" (UID: "f64a3896-9044-479f-9500-65802339e7a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 19:21:26.077666 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:21:26.077591 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2\": container with ID starting with 4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2 not found: ID does not exist" containerID="4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2" Sep 13 19:21:26.077666 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.077627 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2"} err="failed to get container status \"4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2\": rpc error: code = NotFound desc = could not find container \"4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2\": container with ID starting with 4f33163882ac300498f39936ba9663f514ca6e9b7568a7aec21c5876c0be7ca2 not found: ID does not exist" Sep 13 19:21:26.079062 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.079025 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f64a3896-9044-479f-9500-65802339e7a6-kube-api-access-jbxbd" (OuterVolumeSpecName: "kube-api-access-jbxbd") pod "f64a3896-9044-479f-9500-65802339e7a6" (UID: "f64a3896-9044-479f-9500-65802339e7a6"). InnerVolumeSpecName "kube-api-access-jbxbd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:21:26.084823 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.084784 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-h84nw"] Sep 13 19:21:26.092819 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.092787 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-h84nw"] Sep 13 19:21:26.178764 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.178664 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbxbd\" (UniqueName: \"kubernetes.io/projected/f64a3896-9044-479f-9500-65802339e7a6-kube-api-access-jbxbd\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:21:26.178764 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.178698 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f64a3896-9044-479f-9500-65802339e7a6-plugins-config-volume\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:21:26.514378 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.514334 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5"] Sep 13 19:21:26.514745 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.514732 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58de5cc7-1e42-4a41-9166-38acf02019d9" containerName="manager" Sep 13 19:21:26.514794 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.514747 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="58de5cc7-1e42-4a41-9166-38acf02019d9" containerName="manager" Sep 13 19:21:26.514794 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.514762 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a347145-36da-4d00-8983-e0bb69801299" containerName="payload-processing" Sep 13 19:21:26.514794 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.514768 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a347145-36da-4d00-8983-e0bb69801299" containerName="payload-processing" Sep 13 19:21:26.514794 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.514776 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f64a3896-9044-479f-9500-65802339e7a6" containerName="payload-pre-processing" Sep 13 19:21:26.514794 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.514781 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f64a3896-9044-479f-9500-65802339e7a6" containerName="payload-pre-processing" Sep 13 19:21:26.514945 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.514874 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="58de5cc7-1e42-4a41-9166-38acf02019d9" containerName="manager" Sep 13 19:21:26.514945 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.514885 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f64a3896-9044-479f-9500-65802339e7a6" containerName="payload-pre-processing" Sep 13 19:21:26.514945 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.514891 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a347145-36da-4d00-8983-e0bb69801299" containerName="payload-processing" Sep 13 19:21:26.519766 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.519735 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" Sep 13 19:21:26.523143 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.522544 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-vgqml\"" Sep 13 19:21:26.523143 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.522818 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\"" Sep 13 19:21:26.526961 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.526928 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5"] Sep 13 19:21:26.583161 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.583120 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hgwp\" (UniqueName: \"kubernetes.io/projected/e2597494-9d91-4e93-93bd-ec630e95516f-kube-api-access-5hgwp\") pod \"ai-gateway-controller-5f6f854ddd-4k2f5\" (UID: \"e2597494-9d91-4e93-93bd-ec630e95516f\") " pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" Sep 13 19:21:26.684155 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.684117 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hgwp\" (UniqueName: \"kubernetes.io/projected/e2597494-9d91-4e93-93bd-ec630e95516f-kube-api-access-5hgwp\") pod \"ai-gateway-controller-5f6f854ddd-4k2f5\" (UID: \"e2597494-9d91-4e93-93bd-ec630e95516f\") " pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" Sep 13 19:21:26.691808 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.691775 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hgwp\" (UniqueName: \"kubernetes.io/projected/e2597494-9d91-4e93-93bd-ec630e95516f-kube-api-access-5hgwp\") pod \"ai-gateway-controller-5f6f854ddd-4k2f5\" (UID: \"e2597494-9d91-4e93-93bd-ec630e95516f\") " pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" Sep 13 19:21:26.736149 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.736100 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-f7659468d-5gg4k" podUID="58de5cc7-1e42-4a41-9166-38acf02019d9" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.47:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 13 19:21:26.835993 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:26.835879 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" Sep 13 19:21:27.049301 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:27.049269 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl" Sep 13 19:21:27.071046 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:27.071004 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl"] Sep 13 19:21:27.074004 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:27.073968 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-4cgwl"] Sep 13 19:21:27.112880 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:27.112852 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5"] Sep 13 19:21:27.114928 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:21:27.114896 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2597494_9d91_4e93_93bd_ec630e95516f.slice/crio-f2d88ea9ac2bd2ac64b23720d316c6def1506a71d706a094481c21159d47b2a3 WatchSource:0}: Error finding container f2d88ea9ac2bd2ac64b23720d316c6def1506a71d706a094481c21159d47b2a3: Status 404 returned error can't find the container with id f2d88ea9ac2bd2ac64b23720d316c6def1506a71d706a094481c21159d47b2a3 Sep 13 19:21:27.133514 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:27.133476 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a347145-36da-4d00-8983-e0bb69801299" path="/var/lib/kubelet/pods/1a347145-36da-4d00-8983-e0bb69801299/volumes" Sep 13 19:21:27.133947 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:27.133927 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58de5cc7-1e42-4a41-9166-38acf02019d9" path="/var/lib/kubelet/pods/58de5cc7-1e42-4a41-9166-38acf02019d9/volumes" Sep 13 19:21:27.134330 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:27.134312 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f64a3896-9044-479f-9500-65802339e7a6" path="/var/lib/kubelet/pods/f64a3896-9044-479f-9500-65802339e7a6/volumes" Sep 13 19:21:28.059842 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.059799 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" event={"ID":"e2597494-9d91-4e93-93bd-ec630e95516f","Type":"ContainerStarted","Data":"f2d88ea9ac2bd2ac64b23720d316c6def1506a71d706a094481c21159d47b2a3"} Sep 13 19:21:28.714933 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.714896 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-784c4d5dc4-qs6n7"] Sep 13 19:21:28.718721 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.718693 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" Sep 13 19:21:28.721219 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.721190 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ztnfv\"" Sep 13 19:21:28.721383 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.721190 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 13 19:21:28.725086 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.725059 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-784c4d5dc4-qs6n7"] Sep 13 19:21:28.743889 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.743851 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7f694cf979-tp4w8"] Sep 13 19:21:28.746564 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.746532 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" Sep 13 19:21:28.755330 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.755298 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7f694cf979-tp4w8"] Sep 13 19:21:28.806603 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.806486 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbb57\" (UniqueName: \"kubernetes.io/projected/f5800954-e2ac-45b1-9ed9-43e497627b5d-kube-api-access-gbb57\") pod \"payload-pre-processing-7f694cf979-tp4w8\" (UID: \"f5800954-e2ac-45b1-9ed9-43e497627b5d\") " pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" Sep 13 19:21:28.806603 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.806578 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw8ds\" (UniqueName: \"kubernetes.io/projected/4f7376bd-c767-4fc8-ac71-5d58d54a0c5a-kube-api-access-pw8ds\") pod \"payload-processing-784c4d5dc4-qs6n7\" (UID: \"4f7376bd-c767-4fc8-ac71-5d58d54a0c5a\") " pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" Sep 13 19:21:28.806857 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.806620 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5800954-e2ac-45b1-9ed9-43e497627b5d-plugins-config-volume\") pod \"payload-pre-processing-7f694cf979-tp4w8\" (UID: \"f5800954-e2ac-45b1-9ed9-43e497627b5d\") " pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" Sep 13 19:21:28.806857 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.806702 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7376bd-c767-4fc8-ac71-5d58d54a0c5a-plugins-config-volume\") pod \"payload-processing-784c4d5dc4-qs6n7\" (UID: \"4f7376bd-c767-4fc8-ac71-5d58d54a0c5a\") " pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" Sep 13 19:21:28.907772 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.907726 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw8ds\" (UniqueName: \"kubernetes.io/projected/4f7376bd-c767-4fc8-ac71-5d58d54a0c5a-kube-api-access-pw8ds\") pod \"payload-processing-784c4d5dc4-qs6n7\" (UID: \"4f7376bd-c767-4fc8-ac71-5d58d54a0c5a\") " pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" Sep 13 19:21:28.907772 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.907780 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5800954-e2ac-45b1-9ed9-43e497627b5d-plugins-config-volume\") pod \"payload-pre-processing-7f694cf979-tp4w8\" (UID: \"f5800954-e2ac-45b1-9ed9-43e497627b5d\") " pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" Sep 13 19:21:28.908009 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.907818 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7376bd-c767-4fc8-ac71-5d58d54a0c5a-plugins-config-volume\") pod \"payload-processing-784c4d5dc4-qs6n7\" (UID: \"4f7376bd-c767-4fc8-ac71-5d58d54a0c5a\") " pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" Sep 13 19:21:28.908009 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.907901 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbb57\" (UniqueName: \"kubernetes.io/projected/f5800954-e2ac-45b1-9ed9-43e497627b5d-kube-api-access-gbb57\") pod \"payload-pre-processing-7f694cf979-tp4w8\" (UID: \"f5800954-e2ac-45b1-9ed9-43e497627b5d\") " pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" Sep 13 19:21:28.909030 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.909003 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7376bd-c767-4fc8-ac71-5d58d54a0c5a-plugins-config-volume\") pod \"payload-processing-784c4d5dc4-qs6n7\" (UID: \"4f7376bd-c767-4fc8-ac71-5d58d54a0c5a\") " pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" Sep 13 19:21:28.909120 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.909003 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5800954-e2ac-45b1-9ed9-43e497627b5d-plugins-config-volume\") pod \"payload-pre-processing-7f694cf979-tp4w8\" (UID: \"f5800954-e2ac-45b1-9ed9-43e497627b5d\") " pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" Sep 13 19:21:28.916968 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.916939 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbb57\" (UniqueName: \"kubernetes.io/projected/f5800954-e2ac-45b1-9ed9-43e497627b5d-kube-api-access-gbb57\") pod \"payload-pre-processing-7f694cf979-tp4w8\" (UID: \"f5800954-e2ac-45b1-9ed9-43e497627b5d\") " pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" Sep 13 19:21:28.917116 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:28.917075 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw8ds\" (UniqueName: \"kubernetes.io/projected/4f7376bd-c767-4fc8-ac71-5d58d54a0c5a-kube-api-access-pw8ds\") pod \"payload-processing-784c4d5dc4-qs6n7\" (UID: \"4f7376bd-c767-4fc8-ac71-5d58d54a0c5a\") " pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" Sep 13 19:21:29.031980 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:29.031937 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" Sep 13 19:21:29.067645 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:29.067545 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" event={"ID":"e2597494-9d91-4e93-93bd-ec630e95516f","Type":"ContainerStarted","Data":"bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734"} Sep 13 19:21:29.068062 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:29.067744 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" Sep 13 19:21:29.072557 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:29.072537 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" Sep 13 19:21:29.082743 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:29.082687 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" podStartSLOduration=1.683607656 podStartE2EDuration="3.082667593s" totalImagesPullingTime="1.399059937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:21:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:21:29.081522368 +0000 UTC m=+1040.463051398" watchObservedRunningTime="2026-09-13 19:21:29.082667593 +0000 UTC m=+1040.464196622" Sep 13 19:21:29.310849 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:29.310824 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-784c4d5dc4-qs6n7"] Sep 13 19:21:29.312986 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:21:29.312960 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7376bd_c767_4fc8_ac71_5d58d54a0c5a.slice/crio-04df9bf8408010851fe1e85e6a9d11f65d76ebad3de496aecb55dcd0f1e01530 WatchSource:0}: Error finding container 04df9bf8408010851fe1e85e6a9d11f65d76ebad3de496aecb55dcd0f1e01530: Status 404 returned error can't find the container with id 04df9bf8408010851fe1e85e6a9d11f65d76ebad3de496aecb55dcd0f1e01530 Sep 13 19:21:29.356343 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:29.356224 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7f694cf979-tp4w8"] Sep 13 19:21:29.360205 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:21:29.360172 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5800954_e2ac_45b1_9ed9_43e497627b5d.slice/crio-3ff79e9f0264e6e72f70f9fe252af16e370c700e780425496dc1264d527bbca2 WatchSource:0}: Error finding container 3ff79e9f0264e6e72f70f9fe252af16e370c700e780425496dc1264d527bbca2: Status 404 returned error can't find the container with id 3ff79e9f0264e6e72f70f9fe252af16e370c700e780425496dc1264d527bbca2 Sep 13 19:21:30.073780 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:30.073743 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" event={"ID":"4f7376bd-c767-4fc8-ac71-5d58d54a0c5a","Type":"ContainerStarted","Data":"04df9bf8408010851fe1e85e6a9d11f65d76ebad3de496aecb55dcd0f1e01530"} Sep 13 19:21:30.074896 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:30.074867 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" event={"ID":"f5800954-e2ac-45b1-9ed9-43e497627b5d","Type":"ContainerStarted","Data":"3ff79e9f0264e6e72f70f9fe252af16e370c700e780425496dc1264d527bbca2"} Sep 13 19:21:31.081548 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:31.081511 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" event={"ID":"f5800954-e2ac-45b1-9ed9-43e497627b5d","Type":"ContainerStarted","Data":"af157a9e285e703aa0d0bfba009710c28dfab9e5534bf46a2389eece5979fe66"} Sep 13 19:21:31.082052 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:31.081602 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" Sep 13 19:21:31.083170 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:31.083141 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" event={"ID":"4f7376bd-c767-4fc8-ac71-5d58d54a0c5a","Type":"ContainerStarted","Data":"7834a471ad391bc067fa4c0d996fad5812f29a852d257fd58fdf38aab9679e27"} Sep 13 19:21:31.083304 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:31.083288 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" Sep 13 19:21:31.096539 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:31.096487 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" podStartSLOduration=1.915587954 podStartE2EDuration="3.096468045s" totalImagesPullingTime="1.180880091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:21:31.094706204 +0000 UTC m=+1042.476235233" watchObservedRunningTime="2026-09-13 19:21:31.096468045 +0000 UTC m=+1042.477997068" Sep 13 19:21:31.108862 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:31.108801 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" podStartSLOduration=1.878522853 podStartE2EDuration="3.108782666s" totalImagesPullingTime="1.230259813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:21:31.107996888 +0000 UTC m=+1042.489525918" watchObservedRunningTime="2026-09-13 19:21:31.108782666 +0000 UTC m=+1042.490311714" Sep 13 19:21:40.077883 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:40.077848 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" Sep 13 19:21:40.268873 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:40.268832 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7f65f67775-lbljc"] Sep 13 19:21:40.272726 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:40.272699 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7f65f67775-lbljc" Sep 13 19:21:40.279594 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:40.279355 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7f65f67775-lbljc"] Sep 13 19:21:40.317513 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:40.317469 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx5kw\" (UniqueName: \"kubernetes.io/projected/a92d20bc-bc12-4da4-9dfe-c731747cd406-kube-api-access-lx5kw\") pod \"ai-gateway-controller-7f65f67775-lbljc\" (UID: \"a92d20bc-bc12-4da4-9dfe-c731747cd406\") " pod="opendatahub/ai-gateway-controller-7f65f67775-lbljc" Sep 13 19:21:40.418808 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:40.418712 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx5kw\" (UniqueName: \"kubernetes.io/projected/a92d20bc-bc12-4da4-9dfe-c731747cd406-kube-api-access-lx5kw\") pod \"ai-gateway-controller-7f65f67775-lbljc\" (UID: \"a92d20bc-bc12-4da4-9dfe-c731747cd406\") " pod="opendatahub/ai-gateway-controller-7f65f67775-lbljc" Sep 13 19:21:40.426880 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:40.426853 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx5kw\" (UniqueName: \"kubernetes.io/projected/a92d20bc-bc12-4da4-9dfe-c731747cd406-kube-api-access-lx5kw\") pod \"ai-gateway-controller-7f65f67775-lbljc\" (UID: \"a92d20bc-bc12-4da4-9dfe-c731747cd406\") " pod="opendatahub/ai-gateway-controller-7f65f67775-lbljc" Sep 13 19:21:40.584744 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:40.584700 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7f65f67775-lbljc" Sep 13 19:21:40.860930 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:40.860901 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7f65f67775-lbljc"] Sep 13 19:21:40.863675 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:21:40.863635 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda92d20bc_bc12_4da4_9dfe_c731747cd406.slice/crio-8aba50e89fe0985db3e29f9e5c56663ab185b20e64e8117a63739e89c138778a WatchSource:0}: Error finding container 8aba50e89fe0985db3e29f9e5c56663ab185b20e64e8117a63739e89c138778a: Status 404 returned error can't find the container with id 8aba50e89fe0985db3e29f9e5c56663ab185b20e64e8117a63739e89c138778a Sep 13 19:21:41.144618 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:41.144512 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7f65f67775-lbljc" event={"ID":"a92d20bc-bc12-4da4-9dfe-c731747cd406","Type":"ContainerStarted","Data":"8aba50e89fe0985db3e29f9e5c56663ab185b20e64e8117a63739e89c138778a"} Sep 13 19:21:42.091376 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:42.091346 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7f694cf979-tp4w8" Sep 13 19:21:42.091578 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:42.091424 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-784c4d5dc4-qs6n7" Sep 13 19:21:42.152291 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:42.152253 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7f65f67775-lbljc" event={"ID":"a92d20bc-bc12-4da4-9dfe-c731747cd406","Type":"ContainerStarted","Data":"c75516ae84cb6da43dfcf0c17b69651c8454826527172cb4774cab4f9f574cce"} Sep 13 19:21:42.152778 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:42.152339 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7f65f67775-lbljc" Sep 13 19:21:42.168312 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:42.168254 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7f65f67775-lbljc" podStartSLOduration=1.757929416 podStartE2EDuration="2.168222226s" totalImagesPullingTime="410.29281ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:21:42.166092722 +0000 UTC m=+1053.547621751" watchObservedRunningTime="2026-09-13 19:21:42.168222226 +0000 UTC m=+1053.549751256" Sep 13 19:21:53.160387 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:53.160355 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7f65f67775-lbljc" Sep 13 19:21:53.210759 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:53.210721 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5"] Sep 13 19:21:53.211061 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:53.211005 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" podUID="e2597494-9d91-4e93-93bd-ec630e95516f" containerName="manager" containerID="cri-o://bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734" gracePeriod=30 Sep 13 19:21:53.470427 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:53.470390 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" Sep 13 19:21:53.653472 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:53.653432 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hgwp\" (UniqueName: \"kubernetes.io/projected/e2597494-9d91-4e93-93bd-ec630e95516f-kube-api-access-5hgwp\") pod \"e2597494-9d91-4e93-93bd-ec630e95516f\" (UID: \"e2597494-9d91-4e93-93bd-ec630e95516f\") " Sep 13 19:21:53.655587 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:53.655550 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2597494-9d91-4e93-93bd-ec630e95516f-kube-api-access-5hgwp" (OuterVolumeSpecName: "kube-api-access-5hgwp") pod "e2597494-9d91-4e93-93bd-ec630e95516f" (UID: "e2597494-9d91-4e93-93bd-ec630e95516f"). InnerVolumeSpecName "kube-api-access-5hgwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 19:21:53.754341 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:53.754297 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hgwp\" (UniqueName: \"kubernetes.io/projected/e2597494-9d91-4e93-93bd-ec630e95516f-kube-api-access-5hgwp\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Sep 13 19:21:54.221594 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.221560 2563 generic.go:356] "Generic (PLEG): container finished" podID="e2597494-9d91-4e93-93bd-ec630e95516f" containerID="bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734" exitCode=0 Sep 13 19:21:54.222055 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.221611 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" Sep 13 19:21:54.222055 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.221629 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" event={"ID":"e2597494-9d91-4e93-93bd-ec630e95516f","Type":"ContainerDied","Data":"bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734"} Sep 13 19:21:54.222055 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.221671 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5" event={"ID":"e2597494-9d91-4e93-93bd-ec630e95516f","Type":"ContainerDied","Data":"f2d88ea9ac2bd2ac64b23720d316c6def1506a71d706a094481c21159d47b2a3"} Sep 13 19:21:54.222055 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.221693 2563 scope.go:117] "RemoveContainer" containerID="bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734" Sep 13 19:21:54.230927 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.230908 2563 scope.go:117] "RemoveContainer" containerID="bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734" Sep 13 19:21:54.231225 ip-10-0-129-160 kubenswrapper[2563]: E0913 19:21:54.231206 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734\": container with ID starting with bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734 not found: ID does not exist" containerID="bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734" Sep 13 19:21:54.231337 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.231314 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734"} err="failed to get container status \"bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734\": rpc error: code = NotFound desc = could not find container \"bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734\": container with ID starting with bcf4cce921072e5c142d6b443f346bc3022e15008570b5bb18637e663b1b7734 not found: ID does not exist" Sep 13 19:21:54.246389 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.246352 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5"] Sep 13 19:21:54.248977 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.248948 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-5f6f854ddd-4k2f5"] Sep 13 19:21:54.736039 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.736002 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f7659468d-5qrgr"] Sep 13 19:21:54.736506 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.736483 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2597494-9d91-4e93-93bd-ec630e95516f" containerName="manager" Sep 13 19:21:54.736506 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.736506 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2597494-9d91-4e93-93bd-ec630e95516f" containerName="manager" Sep 13 19:21:54.736691 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.736619 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2597494-9d91-4e93-93bd-ec630e95516f" containerName="manager" Sep 13 19:21:54.740759 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.740738 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:21:54.743345 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.743319 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 13 19:21:54.744349 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.744326 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 13 19:21:54.744487 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.744328 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9l67n\"" Sep 13 19:21:54.744487 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.744368 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 13 19:21:54.748906 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.748874 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f7659468d-5qrgr"] Sep 13 19:21:54.866032 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.865996 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssfpw\" (UniqueName: \"kubernetes.io/projected/c50c6b41-0205-431a-95f2-f1ad6425dc5d-kube-api-access-ssfpw\") pod \"maas-controller-f7659468d-5qrgr\" (UID: \"c50c6b41-0205-431a-95f2-f1ad6425dc5d\") " pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:21:54.866032 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.866036 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c50c6b41-0205-431a-95f2-f1ad6425dc5d-metrics-certs\") pod \"maas-controller-f7659468d-5qrgr\" (UID: \"c50c6b41-0205-431a-95f2-f1ad6425dc5d\") " pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:21:54.866291 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.866060 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c50c6b41-0205-431a-95f2-f1ad6425dc5d-webhook-cert\") pod \"maas-controller-f7659468d-5qrgr\" (UID: \"c50c6b41-0205-431a-95f2-f1ad6425dc5d\") " pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:21:54.967108 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.967068 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssfpw\" (UniqueName: \"kubernetes.io/projected/c50c6b41-0205-431a-95f2-f1ad6425dc5d-kube-api-access-ssfpw\") pod \"maas-controller-f7659468d-5qrgr\" (UID: \"c50c6b41-0205-431a-95f2-f1ad6425dc5d\") " pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:21:54.967108 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.967114 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c50c6b41-0205-431a-95f2-f1ad6425dc5d-metrics-certs\") pod \"maas-controller-f7659468d-5qrgr\" (UID: \"c50c6b41-0205-431a-95f2-f1ad6425dc5d\") " pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:21:54.967360 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.967138 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c50c6b41-0205-431a-95f2-f1ad6425dc5d-webhook-cert\") pod \"maas-controller-f7659468d-5qrgr\" (UID: \"c50c6b41-0205-431a-95f2-f1ad6425dc5d\") " pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:21:54.969817 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.969791 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c50c6b41-0205-431a-95f2-f1ad6425dc5d-webhook-cert\") pod \"maas-controller-f7659468d-5qrgr\" (UID: \"c50c6b41-0205-431a-95f2-f1ad6425dc5d\") " pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:21:54.969925 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.969815 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c50c6b41-0205-431a-95f2-f1ad6425dc5d-metrics-certs\") pod \"maas-controller-f7659468d-5qrgr\" (UID: \"c50c6b41-0205-431a-95f2-f1ad6425dc5d\") " pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:21:54.974176 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:54.974154 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssfpw\" (UniqueName: \"kubernetes.io/projected/c50c6b41-0205-431a-95f2-f1ad6425dc5d-kube-api-access-ssfpw\") pod \"maas-controller-f7659468d-5qrgr\" (UID: \"c50c6b41-0205-431a-95f2-f1ad6425dc5d\") " pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:21:55.053088 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:55.052973 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:21:55.132404 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:55.132373 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2597494-9d91-4e93-93bd-ec630e95516f" path="/var/lib/kubelet/pods/e2597494-9d91-4e93-93bd-ec630e95516f/volumes" Sep 13 19:21:55.329052 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:55.329025 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f7659468d-5qrgr"] Sep 13 19:21:55.331168 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:21:55.331140 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc50c6b41_0205_431a_95f2_f1ad6425dc5d.slice/crio-a281867685c5e4fce58d383e089d9b5adb140beded2601206ea6fd557be0aa2e WatchSource:0}: Error finding container a281867685c5e4fce58d383e089d9b5adb140beded2601206ea6fd557be0aa2e: Status 404 returned error can't find the container with id a281867685c5e4fce58d383e089d9b5adb140beded2601206ea6fd557be0aa2e Sep 13 19:21:56.236735 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:56.236694 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f7659468d-5qrgr" event={"ID":"c50c6b41-0205-431a-95f2-f1ad6425dc5d","Type":"ContainerStarted","Data":"554626bf8e5182c0c6fde1bfc52612ddaf9368171f17ce7153e8300d2eb34f65"} Sep 13 19:21:56.236931 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:56.236744 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f7659468d-5qrgr" event={"ID":"c50c6b41-0205-431a-95f2-f1ad6425dc5d","Type":"ContainerStarted","Data":"a281867685c5e4fce58d383e089d9b5adb140beded2601206ea6fd557be0aa2e"} Sep 13 19:21:56.236931 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:56.236775 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:21:56.252512 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:21:56.252450 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f7659468d-5qrgr" podStartSLOduration=1.860475638 podStartE2EDuration="2.252425247s" totalImagesPullingTime="391.949609ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:21:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:21:56.251941653 +0000 UTC m=+1067.633470683" watchObservedRunningTime="2026-09-13 19:21:56.252425247 +0000 UTC m=+1067.633954275" Sep 13 19:22:07.244224 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:07.244139 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f7659468d-5qrgr" Sep 13 19:22:19.890707 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:19.890669 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-68dqb/must-gather-8ccnf"] Sep 13 19:22:19.894708 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:19.894684 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-68dqb/must-gather-8ccnf" Sep 13 19:22:19.896992 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:19.896967 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-68dqb\"/\"openshift-service-ca.crt\"" Sep 13 19:22:19.897129 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:19.896998 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-68dqb\"/\"kube-root-ca.crt\"" Sep 13 19:22:19.897129 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:19.897068 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-68dqb\"/\"default-dockercfg-l8csm\"" Sep 13 19:22:19.902466 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:19.902436 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-68dqb/must-gather-8ccnf"] Sep 13 19:22:20.002686 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:20.002637 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/14670e55-0e60-4fec-a8d1-5a8490244f9c-must-gather-output\") pod \"must-gather-8ccnf\" (UID: \"14670e55-0e60-4fec-a8d1-5a8490244f9c\") " pod="openshift-must-gather-68dqb/must-gather-8ccnf" Sep 13 19:22:20.002686 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:20.002693 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmzs9\" (UniqueName: \"kubernetes.io/projected/14670e55-0e60-4fec-a8d1-5a8490244f9c-kube-api-access-cmzs9\") pod \"must-gather-8ccnf\" (UID: \"14670e55-0e60-4fec-a8d1-5a8490244f9c\") " pod="openshift-must-gather-68dqb/must-gather-8ccnf" Sep 13 19:22:20.103679 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:20.103640 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/14670e55-0e60-4fec-a8d1-5a8490244f9c-must-gather-output\") pod \"must-gather-8ccnf\" (UID: \"14670e55-0e60-4fec-a8d1-5a8490244f9c\") " pod="openshift-must-gather-68dqb/must-gather-8ccnf" Sep 13 19:22:20.103679 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:20.103688 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmzs9\" (UniqueName: \"kubernetes.io/projected/14670e55-0e60-4fec-a8d1-5a8490244f9c-kube-api-access-cmzs9\") pod \"must-gather-8ccnf\" (UID: \"14670e55-0e60-4fec-a8d1-5a8490244f9c\") " pod="openshift-must-gather-68dqb/must-gather-8ccnf" Sep 13 19:22:20.104057 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:20.104035 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/14670e55-0e60-4fec-a8d1-5a8490244f9c-must-gather-output\") pod \"must-gather-8ccnf\" (UID: \"14670e55-0e60-4fec-a8d1-5a8490244f9c\") " pod="openshift-must-gather-68dqb/must-gather-8ccnf" Sep 13 19:22:20.116057 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:20.116025 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmzs9\" (UniqueName: \"kubernetes.io/projected/14670e55-0e60-4fec-a8d1-5a8490244f9c-kube-api-access-cmzs9\") pod \"must-gather-8ccnf\" (UID: \"14670e55-0e60-4fec-a8d1-5a8490244f9c\") " pod="openshift-must-gather-68dqb/must-gather-8ccnf" Sep 13 19:22:20.206172 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:20.206067 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-68dqb/must-gather-8ccnf" Sep 13 19:22:20.479880 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:20.479849 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-68dqb/must-gather-8ccnf"] Sep 13 19:22:20.481906 ip-10-0-129-160 kubenswrapper[2563]: W0913 19:22:20.481873 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14670e55_0e60_4fec_a8d1_5a8490244f9c.slice/crio-ba218cfba41e7a5f3c99aa70dff326acb61c7a1c2c88f84b07207dffea8a1d38 WatchSource:0}: Error finding container ba218cfba41e7a5f3c99aa70dff326acb61c7a1c2c88f84b07207dffea8a1d38: Status 404 returned error can't find the container with id ba218cfba41e7a5f3c99aa70dff326acb61c7a1c2c88f84b07207dffea8a1d38 Sep 13 19:22:21.382826 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:21.382788 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-68dqb/must-gather-8ccnf" event={"ID":"14670e55-0e60-4fec-a8d1-5a8490244f9c","Type":"ContainerStarted","Data":"ba218cfba41e7a5f3c99aa70dff326acb61c7a1c2c88f84b07207dffea8a1d38"} Sep 13 19:22:22.394458 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:22.394409 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-68dqb/must-gather-8ccnf" event={"ID":"14670e55-0e60-4fec-a8d1-5a8490244f9c","Type":"ContainerStarted","Data":"951614b759d01f03b47190db813007c7de63e106330b88259f4e5a3f1e32e3ca"} Sep 13 19:22:22.394968 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:22.394479 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-68dqb/must-gather-8ccnf" event={"ID":"14670e55-0e60-4fec-a8d1-5a8490244f9c","Type":"ContainerStarted","Data":"36399252a539ebb435750a707a4af64dc11099fe0c7542a3d00fc7afba747d57"} Sep 13 19:22:22.411349 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:22.411276 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-68dqb/must-gather-8ccnf" podStartSLOduration=2.6031899259999998 podStartE2EDuration="3.411230082s" totalImagesPullingTime="808.040156ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 19:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:22:22.410595706 +0000 UTC m=+1093.792124735" watchObservedRunningTime="2026-09-13 19:22:22.411230082 +0000 UTC m=+1093.792759111" Sep 13 19:22:22.886050 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:22.886021 2563 ???:1] "http: TLS handshake error from 10.0.129.197:48114: EOF" Sep 13 19:22:22.891783 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:22.891753 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nf6g7_af6ceb0a-0f1f-42f9-8328-dc6263f46fe2/global-pull-secret-syncer/0.log" Sep 13 19:22:23.027705 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:23.027660 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fflnp_8738a08c-3f02-4fa0-b77c-51357c3eb253/konnectivity-agent/0.log" Sep 13 19:22:23.050864 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:23.050832 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-160.ec2.internal_0df2195e00f1241c11a998a6a4518995/haproxy/0.log" Sep 13 19:22:27.769790 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:27.769750 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c5c57c68f-tbxgn_871de01b-61f4-4b1e-a2a3-771b6f33af1e/authorino/0.log" Sep 13 19:22:27.832308 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:27.832273 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-clzqv_cf7fa383-b652-4a0a-9aec-c871900652a5/manager/0.log" Sep 13 19:22:27.858923 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:27.858874 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-h5bgq_b44afc5d-e7a1-4c18-a155-42eb4f466ba0/manager/0.log" Sep 13 19:22:27.884719 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:27.884687 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-7h6x4_a18120ff-b7bb-4165-bd40-34de6d76dda8/kuadrant-console-plugin/0.log" Sep 13 19:22:27.976867 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:27.976821 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-76p69_6689cd67-1c0f-44ea-8149-76859b424e83/manager/0.log" Sep 13 19:22:29.515258 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:29.515204 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a/alertmanager/0.log" Sep 13 19:22:29.545547 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:29.545506 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a/config-reloader/0.log" Sep 13 19:22:29.568045 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:29.568008 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a/kube-rbac-proxy-web/0.log" Sep 13 19:22:29.592782 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:29.592748 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a/kube-rbac-proxy/0.log" Sep 13 19:22:29.612072 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:29.612041 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a/kube-rbac-proxy-metric/0.log" Sep 13 19:22:29.634185 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:29.634159 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a/prom-label-proxy/0.log" Sep 13 19:22:29.654360 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:29.654317 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55e57fc9-9a30-4b6a-a8a8-2c0efefc3c5a/init-config-reloader/0.log" Sep 13 19:22:29.692588 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:29.692558 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-726sd_6e970a71-b45e-4fdc-a726-6a1e3b97e91c/cluster-monitoring-operator/0.log" Sep 13 19:22:29.777019 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:29.776933 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-c95ff8bbc-2vm78_cdbe965a-b0a3-4cfc-8312-b5e091f80ad9/metrics-server/0.log" Sep 13 19:22:29.975626 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:29.975590 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lfttb_191a7f56-c678-4efa-8f90-8e1d6016085b/node-exporter/0.log" Sep 13 19:22:29.996681 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:29.996649 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lfttb_191a7f56-c678-4efa-8f90-8e1d6016085b/kube-rbac-proxy/0.log" Sep 13 19:22:30.016364 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:30.016334 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lfttb_191a7f56-c678-4efa-8f90-8e1d6016085b/init-textfile/0.log" Sep 13 19:22:30.342804 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:30.342752 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-fffc965b6-pqs8x_5807642b-062c-4a3f-ae82-374a306acd53/telemeter-client/0.log" Sep 13 19:22:30.362764 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:30.362702 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-fffc965b6-pqs8x_5807642b-062c-4a3f-ae82-374a306acd53/reload/0.log" Sep 13 19:22:30.382346 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:30.382311 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-fffc965b6-pqs8x_5807642b-062c-4a3f-ae82-374a306acd53/kube-rbac-proxy/0.log" Sep 13 19:22:32.819300 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:32.819270 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/1.log" Sep 13 19:22:32.823384 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:32.823351 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-875dw_892493a1-d1c1-4f71-9621-77beb61a4a95/console-operator/2.log" Sep 13 19:22:33.419708 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:33.419672 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-sx5m5_066e97f0-a26a-4c7a-a65f-ac4cdf7ca8c0/download-server/0.log" Sep 13 19:22:33.768287 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:33.768220 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94"] Sep 13 19:22:33.775188 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:33.775152 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:33.778172 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:33.778135 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94"] Sep 13 19:22:33.963991 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:33.963952 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d7487b5-5051-475c-bc7f-7f4af28e1809-lib-modules\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:33.964474 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:33.963998 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d7487b5-5051-475c-bc7f-7f4af28e1809-proc\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:33.964474 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:33.964140 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv7n6\" (UniqueName: \"kubernetes.io/projected/5d7487b5-5051-475c-bc7f-7f4af28e1809-kube-api-access-jv7n6\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:33.964474 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:33.964215 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d7487b5-5051-475c-bc7f-7f4af28e1809-podres\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:33.964474 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:33.964310 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d7487b5-5051-475c-bc7f-7f4af28e1809-sys\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:34.065881 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.065778 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv7n6\" (UniqueName: \"kubernetes.io/projected/5d7487b5-5051-475c-bc7f-7f4af28e1809-kube-api-access-jv7n6\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:34.066065 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.065899 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d7487b5-5051-475c-bc7f-7f4af28e1809-podres\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:34.066065 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.065948 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d7487b5-5051-475c-bc7f-7f4af28e1809-sys\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:34.066065 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.066014 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d7487b5-5051-475c-bc7f-7f4af28e1809-lib-modules\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:34.066065 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.066041 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d7487b5-5051-475c-bc7f-7f4af28e1809-proc\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:34.066301 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.066225 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d7487b5-5051-475c-bc7f-7f4af28e1809-sys\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:34.066301 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.066229 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d7487b5-5051-475c-bc7f-7f4af28e1809-proc\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:34.066407 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.066355 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d7487b5-5051-475c-bc7f-7f4af28e1809-podres\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:34.066407 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.066363 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d7487b5-5051-475c-bc7f-7f4af28e1809-lib-modules\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:34.075077 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.074972 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv7n6\" (UniqueName: \"kubernetes.io/projected/5d7487b5-5051-475c-bc7f-7f4af28e1809-kube-api-access-jv7n6\") pod \"perf-node-gather-daemonset-pmd94\" (UID: \"5d7487b5-5051-475c-bc7f-7f4af28e1809\") " pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:34.090700 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.090658 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:34.393188 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.393159 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94"] Sep 13 19:22:34.498461 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:34.498418 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" event={"ID":"5d7487b5-5051-475c-bc7f-7f4af28e1809","Type":"ContainerStarted","Data":"b91c68fdcadbc25fec597aaf56acb1a97a5e6987610f6367fcf3047cd29f44c2"} Sep 13 19:22:35.448734 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:35.448697 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bvg5t_b17a3853-b777-4980-9a42-0e87a2c963f8/dns/0.log" Sep 13 19:22:35.470624 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:35.470593 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bvg5t_b17a3853-b777-4980-9a42-0e87a2c963f8/kube-rbac-proxy/0.log" Sep 13 19:22:35.508183 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:35.508136 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" event={"ID":"5d7487b5-5051-475c-bc7f-7f4af28e1809","Type":"ContainerStarted","Data":"df1620639e4a64dededb65d776c8f3dc92ff7ce75a5a8f47d7f1e79215774883"} Sep 13 19:22:35.509131 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:35.509094 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:35.524701 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:35.524641 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" podStartSLOduration=2.52461958 podStartE2EDuration="2.52461958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 19:22:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 19:22:35.523035657 +0000 UTC m=+1106.904564689" watchObservedRunningTime="2026-09-13 19:22:35.52461958 +0000 UTC m=+1106.906148608" Sep 13 19:22:35.627352 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:35.627320 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ttddh_31fa23e6-3a87-4f8f-9f54-48853123eb54/dns-node-resolver/0.log" Sep 13 19:22:36.195956 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:36.195920 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5569d8f464-5dzh9_1c10e26c-528d-4f49-806d-9c15d418e11c/registry/0.log" Sep 13 19:22:36.213530 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:36.213503 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fbmxs_effec463-bf64-4a1c-b7ad-9bee7e4901df/node-ca/0.log" Sep 13 19:22:37.148427 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:37.148395 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r_f45303d0-58d5-4a0d-890f-c9893e7d28d7/istio-proxy/0.log" Sep 13 19:22:37.261006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:37.260971 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56c745dfd9-blnv6_d18e47df-2426-4e8a-92c7-df3d0339fa1b/kube-auth-proxy/0.log" Sep 13 19:22:37.304678 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:37.304628 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b9dvd_ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb/istio-proxy/0.log" Sep 13 19:22:37.306608 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:37.306570 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b9dvd_ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb/istio-proxy/1.log" Sep 13 19:22:37.333847 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:37.333816 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7f694cf979-tp4w8_f5800954-e2ac-45b1-9ed9-43e497627b5d/payload-pre-processing/0.log" Sep 13 19:22:37.361624 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:37.361594 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-784c4d5dc4-qs6n7_4f7376bd-c767-4fc8-ac71-5d58d54a0c5a/payload-processing/0.log" Sep 13 19:22:38.019925 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:38.019876 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4zfmc_bc0cbcda-2f94-4fd9-bfb9-f2d4454a9327/serve-healthcheck-canary/0.log" Sep 13 19:22:38.622593 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:38.622558 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lkmtm_64b72404-61ee-45b3-bfcd-fcf8a166664b/kube-rbac-proxy/0.log" Sep 13 19:22:38.643896 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:38.643851 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lkmtm_64b72404-61ee-45b3-bfcd-fcf8a166664b/exporter/0.log" Sep 13 19:22:38.663029 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:38.662996 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lkmtm_64b72404-61ee-45b3-bfcd-fcf8a166664b/extractor/0.log" Sep 13 19:22:40.920282 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:40.920232 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7f65f67775-lbljc_a92d20bc-bc12-4da4-9dfe-c731747cd406/manager/0.log" Sep 13 19:22:40.943072 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:40.943029 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-98n7v_094e442e-e861-4888-b314-9c6233b4b321/manager/0.log" Sep 13 19:22:40.985999 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:40.985961 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-f4xt4_698395ff-2ac6-4085-98b7-d702909cfff6/manager/0.log" Sep 13 19:22:41.013742 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:41.013714 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f7659468d-5qrgr_c50c6b41-0205-431a-95f2-f1ad6425dc5d/manager/0.log" Sep 13 19:22:41.100965 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:41.100925 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6gffv_582cefd2-87ac-4225-b77e-26df9795f169/manager/0.log" Sep 13 19:22:41.222518 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:41.222486 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65d87bcbcb-zm5vn_61febc46-e159-491d-bf8d-8b7dc58175bd/manager/0.log" Sep 13 19:22:41.240128 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:41.240083 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65d87bcbcb-zm5vn_61febc46-e159-491d-bf8d-8b7dc58175bd/copy-manifests/0.log" Sep 13 19:22:42.542203 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:42.541432 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-68dqb/perf-node-gather-daemonset-pmd94" Sep 13 19:22:44.635720 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:44.635690 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r_f45303d0-58d5-4a0d-890f-c9893e7d28d7/istio-proxy/0.log" Sep 13 19:22:44.699440 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:44.699396 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56c745dfd9-blnv6_d18e47df-2426-4e8a-92c7-df3d0339fa1b/kube-auth-proxy/0.log" Sep 13 19:22:44.721197 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:44.721159 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b9dvd_ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb/istio-proxy/0.log" Sep 13 19:22:44.725334 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:44.725299 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b9dvd_ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb/istio-proxy/1.log" Sep 13 19:22:44.737558 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:44.737533 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7f694cf979-tp4w8_f5800954-e2ac-45b1-9ed9-43e497627b5d/payload-pre-processing/0.log" Sep 13 19:22:44.753801 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:44.753697 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-784c4d5dc4-qs6n7_4f7376bd-c767-4fc8-ac71-5d58d54a0c5a/payload-processing/0.log" Sep 13 19:22:45.998485 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:45.998453 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hpw5r_f45303d0-58d5-4a0d-890f-c9893e7d28d7/istio-proxy/0.log" Sep 13 19:22:46.065042 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:46.065010 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56c745dfd9-blnv6_d18e47df-2426-4e8a-92c7-df3d0339fa1b/kube-auth-proxy/0.log" Sep 13 19:22:46.088192 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:46.088163 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b9dvd_ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb/istio-proxy/0.log" Sep 13 19:22:46.089371 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:46.089347 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b9dvd_ef7b7dc3-2448-4ef9-8198-c6e6b12fb1bb/istio-proxy/1.log" Sep 13 19:22:46.102006 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:46.101976 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7f694cf979-tp4w8_f5800954-e2ac-45b1-9ed9-43e497627b5d/payload-pre-processing/0.log" Sep 13 19:22:46.116540 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:46.116505 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-784c4d5dc4-qs6n7_4f7376bd-c767-4fc8-ac71-5d58d54a0c5a/payload-processing/0.log" Sep 13 19:22:48.431199 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:48.431104 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-nsbnf_cc5e33f3-8e38-43dc-9f04-bef15c0d860c/kube-storage-version-migrator-operator/1.log" Sep 13 19:22:48.433277 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:48.433246 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-nsbnf_cc5e33f3-8e38-43dc-9f04-bef15c0d860c/kube-storage-version-migrator-operator/0.log" Sep 13 19:22:49.597932 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:49.597899 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fztdh_60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c/kube-multus-additional-cni-plugins/0.log" Sep 13 19:22:49.615374 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:49.615336 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fztdh_60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c/egress-router-binary-copy/0.log" Sep 13 19:22:49.632291 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:49.632233 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fztdh_60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c/cni-plugins/0.log" Sep 13 19:22:49.655049 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:49.655018 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fztdh_60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c/bond-cni-plugin/0.log" Sep 13 19:22:49.672692 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:49.672662 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fztdh_60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c/routeoverride-cni/0.log" Sep 13 19:22:49.690406 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:49.690366 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fztdh_60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c/whereabouts-cni-bincopy/0.log" Sep 13 19:22:49.707152 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:49.707117 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fztdh_60e1f7fc-ed12-4ed7-bb9c-bcceeca0533c/whereabouts-cni/0.log" Sep 13 19:22:49.904794 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:49.904712 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lk24l_56671f6f-f921-4342-a427-bb5d123e57a9/kube-multus/0.log" Sep 13 19:22:49.924674 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:49.924620 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2wc2z_526ccac7-e62a-45c1-bc16-1c2f1bc7aca5/network-metrics-daemon/0.log" Sep 13 19:22:49.938736 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:49.938703 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2wc2z_526ccac7-e62a-45c1-bc16-1c2f1bc7aca5/kube-rbac-proxy/0.log" Sep 13 19:22:50.991716 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:50.991683 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tpxtd_6b418bcb-8317-400a-adde-d604e722d1b8/ovn-controller/0.log" Sep 13 19:22:51.005832 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:51.005801 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tpxtd_6b418bcb-8317-400a-adde-d604e722d1b8/ovn-acl-logging/0.log" Sep 13 19:22:51.032278 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:51.032221 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tpxtd_6b418bcb-8317-400a-adde-d604e722d1b8/kube-rbac-proxy-node/0.log" Sep 13 19:22:51.054914 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:51.054877 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tpxtd_6b418bcb-8317-400a-adde-d604e722d1b8/kube-rbac-proxy-ovn-metrics/0.log" Sep 13 19:22:51.069785 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:51.069753 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tpxtd_6b418bcb-8317-400a-adde-d604e722d1b8/northd/0.log" Sep 13 19:22:51.087503 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:51.087471 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tpxtd_6b418bcb-8317-400a-adde-d604e722d1b8/nbdb/0.log" Sep 13 19:22:51.106306 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:51.106274 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tpxtd_6b418bcb-8317-400a-adde-d604e722d1b8/sbdb/0.log" Sep 13 19:22:51.169123 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:51.169088 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tpxtd_6b418bcb-8317-400a-adde-d604e722d1b8/ovnkube-controller/0.log" Sep 13 19:22:52.384021 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:52.383991 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-6f8b768455-q48gb_b4d96614-aac8-4a7e-86b1-007714a7a27f/check-endpoints/0.log" Sep 13 19:22:52.403631 ip-10-0-129-160 kubenswrapper[2563]: I0913 19:22:52.403596 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-525rm_e9f0b4f6-e276-412b-9055-0389f82519db/network-check-target-container/0.log"