Sep 09 23:15:14.809432 ip-10-0-130-56 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 09 23:15:14.809445 ip-10-0-130-56 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 09 23:15:14.809455 ip-10-0-130-56 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 09 23:15:14.809659 ip-10-0-130-56 systemd[1]: Failed to start Kubernetes Kubelet. Sep 09 23:15:24.868192 ip-10-0-130-56 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 09 23:15:24.868211 ip-10-0-130-56 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 8787455161d44090bc455b435d617999 -- Sep 09 23:17:18.460259 ip-10-0-130-56 systemd[1]: Starting Kubernetes Kubelet... Sep 09 23:17:18.864265 ip-10-0-130-56 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 23:17:18.864265 ip-10-0-130-56 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 23:17:18.864265 ip-10-0-130-56 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 23:17:18.864265 ip-10-0-130-56 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 23:17:18.864265 ip-10-0-130-56 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 23:17:18.866509 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.866413 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 23:17:18.871855 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871827 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 23:17:18.871855 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871847 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 23:17:18.871855 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871850 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 23:17:18.871855 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871854 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 23:17:18.871855 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871857 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 23:17:18.871855 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871860 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 23:17:18.871855 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871863 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 23:17:18.871855 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871866 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871869 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871872 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871875 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871878 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871881 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871883 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871886 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871888 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871891 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871894 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871896 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871899 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871901 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871904 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871906 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871909 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871912 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871915 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871918 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 23:17:18.872152 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871920 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871923 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871925 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871927 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871930 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871933 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871935 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871938 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871940 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871943 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871946 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871948 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871951 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871954 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871957 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871960 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871977 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871980 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871985 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 23:17:18.872632 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871989 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871992 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871995 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.871998 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872001 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872004 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872006 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872009 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872011 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872015 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872019 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872023 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872026 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872029 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872031 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872034 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872036 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872039 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872041 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 23:17:18.873189 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872044 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872047 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872051 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872054 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872057 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872061 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872064 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872067 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872070 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872073 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872075 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872078 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872080 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872083 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872087 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872089 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872092 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872094 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872098 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 23:17:18.873657 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872101 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872104 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872106 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872109 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872111 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872114 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872117 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872119 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872122 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872124 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872127 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872129 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872132 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872134 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872137 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872869 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872876 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872879 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872883 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 23:17:18.874180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872886 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872889 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872891 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872894 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872897 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872901 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872905 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872908 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872911 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872914 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872916 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872919 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872921 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872924 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872927 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872930 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872933 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872935 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872938 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 23:17:18.874648 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872941 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872944 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872946 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872949 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872951 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872954 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872957 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872959 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872984 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872990 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872993 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.872996 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873000 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873004 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873006 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873009 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873012 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873014 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873019 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 23:17:18.875182 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873022 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873025 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873029 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873032 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873035 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873038 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873041 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873044 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873047 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873050 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873052 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873055 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873057 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873060 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873062 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873065 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873067 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873070 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873072 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873075 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873077 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 23:17:18.875667 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873080 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873083 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873085 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873087 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873091 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873094 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873097 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873099 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873102 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873105 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873107 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873109 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873112 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873114 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873117 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873119 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873122 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873124 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873127 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 23:17:18.876196 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873129 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873135 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873138 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873141 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873144 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873146 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873149 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873152 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873154 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873156 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873159 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873162 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873164 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873167 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873169 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873172 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873175 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873559 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873564 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 23:17:18.876670 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873567 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873569 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873572 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873575 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873579 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873583 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873586 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873589 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873592 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873595 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873597 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873600 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873603 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873606 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873610 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873613 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873615 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873618 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873620 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 23:17:18.877164 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873623 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873625 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873628 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873630 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873632 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873635 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873638 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873640 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873643 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873646 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873648 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873651 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873654 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873657 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873662 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873665 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873668 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873671 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873673 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 23:17:18.877625 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873676 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873679 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873682 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873684 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873687 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873690 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873694 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873697 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873699 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873702 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873705 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873708 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873711 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873714 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873716 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873719 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873722 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873726 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873729 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873732 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 23:17:18.878111 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873735 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873737 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873740 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873742 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873745 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873748 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873751 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873753 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873756 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873758 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873761 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873764 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873766 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873769 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873771 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873773 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873776 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873778 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873781 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873783 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 23:17:18.878587 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873786 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873788 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873791 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873794 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873796 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873799 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873801 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873803 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873806 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873808 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873811 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873813 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873816 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873818 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873821 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873823 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873825 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873828 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.873831 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.873911 2580 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.873945 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 23:17:18.879092 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.873952 2580 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.873956 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.873960 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.873982 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.873989 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.873993 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.873996 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.873999 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874002 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874005 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874008 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874012 2580 flags.go:64] FLAG: --cgroup-root="" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874014 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874017 2580 flags.go:64] FLAG: --client-ca-file="" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874022 2580 flags.go:64] FLAG: --cloud-provider="external" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874025 2580 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874030 2580 flags.go:64] FLAG: --cluster-domain="" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874033 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874036 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874039 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874042 2580 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874046 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874049 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 23:17:18.879686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874052 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874055 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874058 2580 flags.go:64] FLAG: --contention-profiling="false" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874061 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874064 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874067 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874070 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874074 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874077 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874080 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874083 2580 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874086 2580 flags.go:64] FLAG: --enable-server="true" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874089 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874093 2580 flags.go:64] FLAG: --event-burst="100" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874096 2580 flags.go:64] FLAG: --event-qps="50" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874099 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874102 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874106 2580 flags.go:64] FLAG: --eviction-hard="" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874109 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874112 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874116 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874119 2580 flags.go:64] FLAG: --eviction-soft="" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874121 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874124 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874127 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 23:17:18.880266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874130 2580 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874133 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874136 2580 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874139 2580 flags.go:64] FLAG: --feature-gates="" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874142 2580 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874145 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874148 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874151 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874154 2580 flags.go:64] FLAG: --healthz-port="10248" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874157 2580 flags.go:64] FLAG: --help="false" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874160 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-130-56.ec2.internal" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874163 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874166 2580 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874169 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874172 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874175 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874178 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874180 2580 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874183 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874186 2580 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874189 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874192 2580 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874195 2580 flags.go:64] FLAG: --kube-reserved="" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874197 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 23:17:18.880913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874201 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874203 2580 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874206 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874209 2580 flags.go:64] FLAG: --lock-file="" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874212 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874215 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874218 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874223 2580 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874226 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874229 2580 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874232 2580 flags.go:64] FLAG: --logging-format="text" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874235 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874238 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874241 2580 flags.go:64] FLAG: --manifest-url="" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874244 2580 flags.go:64] FLAG: --manifest-url-header="" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874248 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874251 2580 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874254 2580 flags.go:64] FLAG: --max-pods="110" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874260 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874263 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874266 2580 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874269 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874272 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874275 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874278 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 23:17:18.881550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874286 2580 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874289 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874292 2580 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874295 2580 flags.go:64] FLAG: --pod-cidr="" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874298 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874303 2580 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874306 2580 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874311 2580 flags.go:64] FLAG: --pods-per-core="0" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874314 2580 flags.go:64] FLAG: --port="10250" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874317 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874320 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a8251f89db042d13" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874323 2580 flags.go:64] FLAG: --qos-reserved="" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874326 2580 flags.go:64] FLAG: --read-only-port="10255" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874329 2580 flags.go:64] FLAG: --register-node="true" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874332 2580 flags.go:64] FLAG: --register-with-taints="" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874335 2580 flags.go:64] FLAG: --registry-burst="10" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874339 2580 flags.go:64] FLAG: --registry-qps="5" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874342 2580 flags.go:64] FLAG: --reserved-cpus="" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874344 2580 flags.go:64] FLAG: --reserved-memory="" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874348 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874351 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874354 2580 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874357 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874360 2580 flags.go:64] FLAG: --runonce="false" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874362 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 23:17:18.882198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874365 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874370 2580 flags.go:64] FLAG: --seccomp-default="false" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874373 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874376 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874379 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874382 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874385 2580 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874388 2580 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874390 2580 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874393 2580 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874396 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874399 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874401 2580 flags.go:64] FLAG: --system-cgroups="" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874404 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874411 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874414 2580 flags.go:64] FLAG: --tls-cert-file="" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874416 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874422 2580 flags.go:64] FLAG: --tls-min-version="" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874425 2580 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874427 2580 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874430 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874433 2580 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874437 2580 flags.go:64] FLAG: --v="2" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874441 2580 flags.go:64] FLAG: --version="false" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874445 2580 flags.go:64] FLAG: --vmodule="" Sep 09 23:17:18.882802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874449 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.874452 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874596 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874600 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874603 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874606 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874609 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874612 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874616 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874620 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874623 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874626 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874629 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874632 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874634 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874637 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874639 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874642 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874644 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874647 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 23:17:18.883818 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874650 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874654 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874656 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874660 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874664 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874667 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874670 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874672 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874675 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874678 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874681 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874684 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874692 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874695 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874698 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874700 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874703 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874705 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874707 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 23:17:18.884427 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874710 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874714 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874716 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874719 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874722 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874724 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874727 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874729 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874732 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874734 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874737 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874739 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874742 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874744 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874748 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874751 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874753 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874756 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874758 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874761 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 23:17:18.884901 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874763 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874766 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874768 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874771 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874774 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874776 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874784 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874787 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874790 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874792 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874795 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874797 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874800 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874804 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874807 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874810 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874812 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874815 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874817 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874820 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 23:17:18.885416 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874822 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874825 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874829 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874833 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874837 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874841 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874846 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874849 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874852 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874854 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874857 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874859 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874862 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874864 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874867 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874869 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874872 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874874 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874877 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 23:17:18.885929 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874880 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874883 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.874885 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.875534 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.882541 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.882558 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.882582 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.882588 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882646 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882654 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882657 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882660 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882663 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882666 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 23:17:18.886430 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882669 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882672 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882676 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882679 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882681 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882684 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882687 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882689 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882692 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882695 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882698 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882700 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882703 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882705 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882708 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882710 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882713 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882715 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882717 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882720 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 23:17:18.886782 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882722 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882725 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882727 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882731 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882734 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882737 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882739 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882742 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882744 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882747 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882750 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882752 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882755 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882757 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882760 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882763 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882765 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882767 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882770 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882773 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 23:17:18.887377 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882775 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882778 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882780 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882783 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882786 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882788 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882791 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882793 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882796 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882798 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882803 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882807 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882809 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882812 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882815 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882819 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882822 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882825 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882828 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882831 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 23:17:18.887890 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882833 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882835 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882838 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882840 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882843 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882845 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882848 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882850 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882853 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882855 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882858 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882860 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882862 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882865 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882868 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882870 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882873 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882875 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882879 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 23:17:18.888392 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882882 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882885 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882888 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882890 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882893 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882896 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882898 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882901 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882904 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882907 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882910 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882912 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882915 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.882917 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.882922 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 23:17:18.888871 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883050 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883055 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883058 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883061 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883064 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883067 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883069 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883072 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883074 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883077 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883081 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883084 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883087 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883090 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883093 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883095 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883098 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883101 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883103 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 23:17:18.889261 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883106 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883108 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883111 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883113 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883116 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883118 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883121 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883124 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883127 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883130 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883132 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883135 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883137 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883140 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883143 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883145 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883149 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883152 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883155 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883158 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 23:17:18.889711 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883161 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883163 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883165 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883168 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883170 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883173 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883175 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883178 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883181 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883183 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883185 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883188 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883190 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883193 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883195 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883198 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883200 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883203 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883205 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883208 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 23:17:18.890227 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883211 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883213 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883216 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883219 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883222 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883224 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883226 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883229 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883231 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883234 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883236 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883239 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883241 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883244 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883246 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883249 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883251 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883253 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883256 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883259 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 23:17:18.890757 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883262 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883264 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883267 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883269 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883272 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883274 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883277 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883279 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883282 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883285 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883288 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883291 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883294 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883296 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883299 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883302 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883304 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883306 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883309 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 23:17:18.891391 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:18.883312 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 23:17:18.892202 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.883317 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 23:17:18.892202 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.883436 2580 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 23:17:18.892202 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.885307 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 23:17:18.892202 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.886346 2580 server.go:1019] "Starting client certificate rotation" Sep 09 23:17:18.892202 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.886449 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 23:17:18.892202 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.886482 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 23:17:18.892202 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.888700 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 23:17:18.892202 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.891421 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 23:17:18.905157 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.905136 2580 log.go:25] "Validated CRI v1 runtime API" Sep 09 23:17:18.908038 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.908019 2580 log.go:25] "Validated CRI v1 image API" Sep 09 23:17:18.909781 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.909759 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 23:17:18.913551 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.913528 2580 fs.go:135] Filesystem UUIDs: map[70001cf1-f09c-4e5c-a85e-bfe55bc1ed6d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9b17ed93-30cf-4879-9942-419c85566718:/dev/nvme0n1p4] Sep 09 23:17:18.913643 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.913550 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 23:17:18.913690 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.913663 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 23:17:18.918901 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.918777 2580 manager.go:216] Machine: {Timestamp:2026-09-09 23:17:18.91747771 +0000 UTC m=+0.343901008 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098544 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2494491b19c25dfbf2ee7925f33796 SystemUUID:ec249449-1b19-c25d-fbf2-ee7925f33796 BootID:87874551-61d4-4090-bc45-5b435d617999 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fc:d0:38:28:fd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fc:d0:38:28:fd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:1b:20:48:12:bc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 23:17:18.918901 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.918886 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 23:17:18.919070 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.918962 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 23:17:18.922181 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.922152 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 23:17:18.922367 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.922184 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-56.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 23:17:18.922993 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.922978 2580 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 23:17:18.923058 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.922999 2580 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 23:17:18.923058 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.923010 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 23:17:18.923736 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.923723 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 23:17:18.925075 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.925064 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 23:17:18.928931 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.928915 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 09 23:17:18.929094 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.929083 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 23:17:18.929152 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.929123 2580 kubelet.go:486] "Attempting to sync node with API server" Sep 09 23:17:18.929152 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.929137 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 23:17:18.929237 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.929155 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 23:17:18.929237 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.929169 2580 kubelet.go:398] "Adding apiserver pod source" Sep 09 23:17:18.929237 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.929181 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 23:17:18.929593 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.929566 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 23:17:18.929663 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.929602 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 23:17:18.934289 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.934271 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 23:17:18.936590 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.936574 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 23:17:18.936688 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.936605 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 23:17:18.936688 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.936658 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l9n7n" Sep 09 23:17:18.936805 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.936760 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 23:17:18.936805 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.936772 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 23:17:18.936805 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.936778 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 23:17:18.936805 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.936784 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 23:17:18.936805 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.936793 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 23:17:18.936805 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.936798 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 23:17:18.937472 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.937462 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 23:17:18.937472 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.937473 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 23:17:18.937537 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.937479 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 23:17:18.937537 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.937492 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 23:17:18.937537 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.937505 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 23:17:18.937537 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.937513 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 23:17:18.938208 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:18.938184 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-56.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 23:17:18.938290 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:18.938184 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 23:17:18.938290 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.938278 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 23:17:18.938350 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.938297 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 23:17:18.942703 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.942689 2580 server.go:1268] "Started kubelet" Sep 09 23:17:18.942847 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.942805 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 23:17:18.942907 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.942874 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 23:17:18.942960 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.942877 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 23:17:18.943687 ip-10-0-130-56 systemd[1]: Started Kubernetes Kubelet. Sep 09 23:17:18.947736 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.947710 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 23:17:18.949259 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.949238 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l9n7n" Sep 09 23:17:18.949474 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:18.949451 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 23:17:18.949760 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.949740 2580 server.go:310] "Adding debug handlers to kubelet server" Sep 09 23:17:18.950908 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.950883 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-56.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 23:17:18.953930 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.953911 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 23:17:18.954581 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.954567 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 23:17:18.954812 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:18.954777 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-56.ec2.internal\" not found" Sep 09 23:17:18.955386 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.955369 2580 factory.go:55] Registering systemd factory Sep 09 23:17:18.955465 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.955389 2580 factory.go:223] Registration of the systemd container factory successfully Sep 09 23:17:18.955529 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.955522 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 23:17:18.955635 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.955611 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 23:17:18.955635 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.955630 2580 factory.go:153] Registering CRI-O factory Sep 09 23:17:18.955635 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.955640 2580 factory.go:223] Registration of the crio container factory successfully Sep 09 23:17:18.955832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.955685 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 23:17:18.955832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.955701 2580 factory.go:103] Registering Raw factory Sep 09 23:17:18.955832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.955631 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 23:17:18.955832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.955716 2580 manager.go:1196] Started watching for new ooms in manager Sep 09 23:17:18.955832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.955808 2580 reconstruct.go:97] "Volume reconstruction finished" Sep 09 23:17:18.955832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.955816 2580 reconciler.go:29] "Reconciler: start to sync state" Sep 09 23:17:18.956130 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.956109 2580 manager.go:349] Starting recovery of all containers Sep 09 23:17:18.959692 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.959661 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 23:17:18.962897 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:18.962873 2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-56.ec2.internal\" not found" node="ip-10-0-130-56.ec2.internal" Sep 09 23:17:18.967519 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.967500 2580 manager.go:354] Recovery completed Sep 09 23:17:18.971996 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.971982 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:17:18.974027 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.974008 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:17:18.974116 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.974041 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:17:18.974116 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.974054 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:17:18.974657 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.974643 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 23:17:18.974726 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.974656 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 23:17:18.974726 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.974659 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:17:18.974726 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.974675 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 09 23:17:18.975749 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.975734 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:17:18.975832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.975767 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:17:18.975832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.975781 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:17:18.976824 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.976811 2580 policy_none.go:49] "None policy: Start" Sep 09 23:17:18.976870 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.976828 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 23:17:18.976870 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.976841 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 23:17:18.978822 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:18.978808 2580 policy_none.go:47] "Start" Sep 09 23:17:19.033032 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.017798 2580 manager.go:337] "Starting Device Plugin manager" Sep 09 23:17:19.033032 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.017863 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 23:17:19.033032 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.017876 2580 server.go:87] "Starting device plugin registration server" Sep 09 23:17:19.033032 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.018370 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 23:17:19.033032 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.018385 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 23:17:19.033032 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.018498 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 23:17:19.033032 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.018583 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 23:17:19.033032 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.018594 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 23:17:19.033032 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.019275 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 23:17:19.033032 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.019321 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-56.ec2.internal\" not found" Sep 09 23:17:19.116405 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.116284 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 23:17:19.117546 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.117521 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 23:17:19.117649 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.117575 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 23:17:19.117649 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.117598 2580 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 23:17:19.117738 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.117706 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 23:17:19.118470 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.118452 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:17:19.119366 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.119347 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:17:19.119443 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.119386 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:17:19.119443 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.119404 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:17:19.119443 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.119438 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.119666 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.119650 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 23:17:19.128126 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.128107 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.128258 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.128133 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-56.ec2.internal\": node \"ip-10-0-130-56.ec2.internal\" not found" Sep 09 23:17:19.140051 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.140023 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-56.ec2.internal\" not found" Sep 09 23:17:19.218189 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.218158 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-56.ec2.internal"] Sep 09 23:17:19.218286 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.218242 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:17:19.219229 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.219213 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:17:19.219357 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.219241 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:17:19.219357 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.219256 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:17:19.221417 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.221405 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:17:19.221555 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.221539 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.221604 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.221571 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:17:19.222190 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.222175 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:17:19.222284 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.222200 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:17:19.222284 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.222209 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:17:19.222284 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.222176 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:17:19.222432 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.222285 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:17:19.222432 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.222303 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:17:19.224102 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.224087 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.224163 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.224118 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 23:17:19.224818 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.224801 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientMemory" Sep 09 23:17:19.224890 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.224828 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 23:17:19.224890 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.224838 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeHasSufficientPID" Sep 09 23:17:19.240668 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.240652 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-56.ec2.internal\" not found" Sep 09 23:17:19.243493 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.243479 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-56.ec2.internal\" not found" node="ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.247517 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.247503 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-56.ec2.internal\" not found" node="ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.257671 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.257651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6bb598d832714d83102c4c0968944793-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal\" (UID: \"6bb598d832714d83102c4c0968944793\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.257742 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.257676 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6bb598d832714d83102c4c0968944793-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal\" (UID: \"6bb598d832714d83102c4c0968944793\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.257742 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.257695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04642d3ff3a391e94be23b8faa39fe5f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-56.ec2.internal\" (UID: \"04642d3ff3a391e94be23b8faa39fe5f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.341122 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.341097 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-56.ec2.internal\" not found" Sep 09 23:17:19.358437 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.358408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6bb598d832714d83102c4c0968944793-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal\" (UID: \"6bb598d832714d83102c4c0968944793\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.358500 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.358456 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6bb598d832714d83102c4c0968944793-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal\" (UID: \"6bb598d832714d83102c4c0968944793\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.358500 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.358473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04642d3ff3a391e94be23b8faa39fe5f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-56.ec2.internal\" (UID: \"04642d3ff3a391e94be23b8faa39fe5f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.358565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.358510 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04642d3ff3a391e94be23b8faa39fe5f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-56.ec2.internal\" (UID: \"04642d3ff3a391e94be23b8faa39fe5f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.358565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.358519 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6bb598d832714d83102c4c0968944793-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal\" (UID: \"6bb598d832714d83102c4c0968944793\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.358565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.358520 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6bb598d832714d83102c4c0968944793-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal\" (UID: \"6bb598d832714d83102c4c0968944793\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.441842 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.441778 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-56.ec2.internal\" not found" Sep 09 23:17:19.542298 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.542264 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-56.ec2.internal\" not found" Sep 09 23:17:19.546400 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.546389 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.550945 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.550923 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-56.ec2.internal" Sep 09 23:17:19.642556 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.642520 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-56.ec2.internal\" not found" Sep 09 23:17:19.743033 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.742952 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-56.ec2.internal\" not found" Sep 09 23:17:19.843504 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.843474 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-56.ec2.internal\" not found" Sep 09 23:17:19.886050 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.886018 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 23:17:19.886558 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.886170 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 23:17:19.886558 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.886196 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 23:17:19.944524 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:19.944487 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-56.ec2.internal\" not found" Sep 09 23:17:19.954312 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.954277 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 23:12:18 +0000 UTC" deadline="2028-04-08 09:43:50.332119233 +0000 UTC" Sep 09 23:17:19.954312 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.954306 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13834h26m30.377815739s" Sep 09 23:17:19.994127 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.994038 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 23:17:19.995568 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:19.995549 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 23:17:20.055608 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.055578 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-56.ec2.internal" Sep 09 23:17:20.068363 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:20.068329 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04642d3ff3a391e94be23b8faa39fe5f.slice/crio-51acb965f7db3574ec177c84b9f6f263181701bf8c33c8623c273ad2312a9f74 WatchSource:0}: Error finding container 51acb965f7db3574ec177c84b9f6f263181701bf8c33c8623c273ad2312a9f74: Status 404 returned error can't find the container with id 51acb965f7db3574ec177c84b9f6f263181701bf8c33c8623c273ad2312a9f74 Sep 09 23:17:20.069040 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.069017 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 23:17:20.070152 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.070139 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" Sep 09 23:17:20.078738 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.078721 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 23:17:20.086646 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:20.086625 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bb598d832714d83102c4c0968944793.slice/crio-42468cf232a449f0074c6dca9a97655c0f87fd00dd664c8e8c3e2c5e3ac0a390 WatchSource:0}: Error finding container 42468cf232a449f0074c6dca9a97655c0f87fd00dd664c8e8c3e2c5e3ac0a390: Status 404 returned error can't find the container with id 42468cf232a449f0074c6dca9a97655c0f87fd00dd664c8e8c3e2c5e3ac0a390 Sep 09 23:17:20.119689 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.119659 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 23:17:20.120800 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.120767 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" event={"ID":"6bb598d832714d83102c4c0968944793","Type":"ContainerStarted","Data":"42468cf232a449f0074c6dca9a97655c0f87fd00dd664c8e8c3e2c5e3ac0a390"} Sep 09 23:17:20.121805 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.121785 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-56.ec2.internal" event={"ID":"04642d3ff3a391e94be23b8faa39fe5f","Type":"ContainerStarted","Data":"51acb965f7db3574ec177c84b9f6f263181701bf8c33c8623c273ad2312a9f74"} Sep 09 23:17:20.763955 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.763916 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 23:17:20.930674 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.930646 2580 apiserver.go:52] "Watching apiserver" Sep 09 23:17:20.939136 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.939097 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 23:17:20.940923 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.940892 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-56.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn","openshift-cluster-node-tuning-operator/tuned-7w29b","openshift-multus/network-metrics-daemon-vdx58","openshift-network-operator/iptables-alerter-v6gvv","kube-system/konnectivity-agent-g4hc4","openshift-dns/node-resolver-fj28m","openshift-image-registry/node-ca-v8n2v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal","openshift-multus/multus-additional-cni-plugins-fbstm","openshift-multus/multus-nw6cg","openshift-network-diagnostics/network-check-target-mr82q","openshift-ovn-kubernetes/ovnkube-node-66lv5","kube-system/global-pull-secret-syncer-xxk7v"] Sep 09 23:17:20.943371 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.943344 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fj28m" Sep 09 23:17:20.945665 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.945646 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.945794 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.945778 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 23:17:20.946038 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.945984 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bxdbl\"" Sep 09 23:17:20.946290 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.946271 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 23:17:20.948291 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.948266 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 23:17:20.948291 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.948290 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 23:17:20.948556 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.948535 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 23:17:20.948661 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.948605 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 23:17:20.948661 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.948654 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 23:17:20.948941 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.948767 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 23:17:20.948941 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.948793 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rcnn6\"" Sep 09 23:17:20.948941 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.948939 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.951128 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.950929 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:20.951128 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:20.951014 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:20.951856 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.951225 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 23:17:20.951856 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.951229 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fs2hq\"" Sep 09 23:17:20.951856 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.951787 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 23:17:20.953296 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.953267 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v6gvv" Sep 09 23:17:20.954384 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.954366 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 23:17:20.955299 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.955284 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 23:17:20.955691 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.955653 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 23:17:20.955771 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.955726 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wfndt\"" Sep 09 23:17:20.955771 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.955751 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 23:17:20.956056 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.955960 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g4hc4" Sep 09 23:17:20.958179 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.958152 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:20.958277 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:20.958221 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:20.958496 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.958480 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 23:17:20.958581 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.958505 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 23:17:20.958581 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.958541 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q2xqz\"" Sep 09 23:17:20.960675 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.960631 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v8n2v" Sep 09 23:17:20.963148 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.963128 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 23:17:20.963489 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.963351 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 23:17:20.963489 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.963398 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8src2\"" Sep 09 23:17:20.963489 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.963437 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 23:17:20.963648 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.963482 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:20.967293 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.965866 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:20.967293 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.966285 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 23:17:20.967293 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.966636 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sbqmf\"" Sep 09 23:17:20.967293 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.966656 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 23:17:20.967293 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.966788 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 23:17:20.967293 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.966887 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 23:17:20.967636 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.967593 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-run\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.967687 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.967658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-lib-modules\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.967733 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.967693 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a8cfb746-8899-4809-b938-560d777b6c85-konnectivity-ca\") pod \"konnectivity-agent-g4hc4\" (UID: \"a8cfb746-8899-4809-b938-560d777b6c85\") " pod="kube-system/konnectivity-agent-g4hc4" Sep 09 23:17:20.967781 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.967729 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b326930-e182-4769-b41a-f79a91a2a5db-kubelet-config\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:20.967781 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.967761 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-run-systemd\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.967887 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.967824 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-etc-openvswitch\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.967887 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.967853 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.967995 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.967903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-sysctl-d\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.968052 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968009 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-kubelet\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.968052 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968039 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-run-ovn\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.968151 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968070 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3132de00-554c-4204-b3f8-446e254f72e7-ovnkube-script-lib\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.968151 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:20.968251 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968141 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-modprobe-d\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.968251 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968201 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b22436d1-37ee-45c9-af11-3ee766cb8e70-hosts-file\") pod \"node-resolver-fj28m\" (UID: \"b22436d1-37ee-45c9-af11-3ee766cb8e70\") " pod="openshift-dns/node-resolver-fj28m" Sep 09 23:17:20.968251 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968234 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-slash\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.968402 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.968402 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968311 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdrhv\" (UniqueName: \"kubernetes.io/projected/c2102a48-7202-4287-81c6-c3a6866ddd11-kube-api-access-wdrhv\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.968402 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968360 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b326930-e182-4769-b41a-f79a91a2a5db-dbus\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:20.968402 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968392 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3132de00-554c-4204-b3f8-446e254f72e7-env-overrides\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.968592 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968423 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6732b2ff-3cde-46f8-a772-8906dbf5a69b-iptables-alerter-script\") pod \"iptables-alerter-v6gvv\" (UID: \"6732b2ff-3cde-46f8-a772-8906dbf5a69b\") " pod="openshift-network-operator/iptables-alerter-v6gvv" Sep 09 23:17:20.968592 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968453 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-var-lib-kubelet\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.968592 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968490 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-kubernetes\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.968592 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968535 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-run-netns\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.968592 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968565 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-var-lib-openvswitch\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.968924 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968660 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-log-socket\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.968924 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968713 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3132de00-554c-4204-b3f8-446e254f72e7-ovnkube-config\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.968924 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968744 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3132de00-554c-4204-b3f8-446e254f72e7-ovn-node-metrics-cert\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.968924 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pstmq\" (UniqueName: \"kubernetes.io/projected/3132de00-554c-4204-b3f8-446e254f72e7-kube-api-access-pstmq\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.968924 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968812 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-sysconfig\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.968924 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-sysctl-conf\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.968924 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968888 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2102a48-7202-4287-81c6-c3a6866ddd11-tmp\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.968924 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968919 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:20.969323 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.968950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b22436d1-37ee-45c9-af11-3ee766cb8e70-tmp-dir\") pod \"node-resolver-fj28m\" (UID: \"b22436d1-37ee-45c9-af11-3ee766cb8e70\") " pod="openshift-dns/node-resolver-fj28m" Sep 09 23:17:20.969323 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.969022 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpxdz\" (UniqueName: \"kubernetes.io/projected/b22436d1-37ee-45c9-af11-3ee766cb8e70-kube-api-access-bpxdz\") pod \"node-resolver-fj28m\" (UID: \"b22436d1-37ee-45c9-af11-3ee766cb8e70\") " pod="openshift-dns/node-resolver-fj28m" Sep 09 23:17:20.969323 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.969105 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-systemd-units\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.969323 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.969142 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd57l\" (UniqueName: \"kubernetes.io/projected/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-kube-api-access-nd57l\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:20.969323 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.969178 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-systemd\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.969323 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.969206 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-sys\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.969323 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.969233 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-run-openvswitch\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.969323 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.969282 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-cni-netd\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.969676 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.969411 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7gxf\" (UniqueName: \"kubernetes.io/projected/6732b2ff-3cde-46f8-a772-8906dbf5a69b-kube-api-access-v7gxf\") pod \"iptables-alerter-v6gvv\" (UID: \"6732b2ff-3cde-46f8-a772-8906dbf5a69b\") " pod="openshift-network-operator/iptables-alerter-v6gvv" Sep 09 23:17:20.969676 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.969575 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-host\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.969676 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.969605 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-tuned\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:20.970042 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.969984 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a8cfb746-8899-4809-b938-560d777b6c85-agent-certs\") pod \"konnectivity-agent-g4hc4\" (UID: \"a8cfb746-8899-4809-b938-560d777b6c85\") " pod="kube-system/konnectivity-agent-g4hc4" Sep 09 23:17:20.970042 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.970025 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-node-log\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.970179 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.970050 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-cni-bin\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:20.970179 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.970100 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6732b2ff-3cde-46f8-a772-8906dbf5a69b-host-slash\") pod \"iptables-alerter-v6gvv\" (UID: \"6732b2ff-3cde-46f8-a772-8906dbf5a69b\") " pod="openshift-network-operator/iptables-alerter-v6gvv" Sep 09 23:17:20.970565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.970316 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 23:17:20.970565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.970355 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 23:17:20.970565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.970421 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 23:17:20.970765 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.970696 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nw6cg" Sep 09 23:17:20.970823 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.970786 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-z8ggh\"" Sep 09 23:17:20.970961 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.970942 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 23:17:20.971078 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.971062 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 23:17:20.972718 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.972698 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 23:17:20.973025 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.973008 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c5sq8\"" Sep 09 23:17:20.973275 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.973255 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:20.973348 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:20.973324 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:20.985039 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.985010 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g5zdr" Sep 09 23:17:20.995538 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:20.995515 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g5zdr" Sep 09 23:17:21.056749 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.056677 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 23:17:21.071240 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071210 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-tuned\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.071240 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071246 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a8cfb746-8899-4809-b938-560d777b6c85-agent-certs\") pod \"konnectivity-agent-g4hc4\" (UID: \"a8cfb746-8899-4809-b938-560d777b6c85\") " pod="kube-system/konnectivity-agent-g4hc4" Sep 09 23:17:21.071484 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071265 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-node-log\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.071484 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071283 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-hostroot\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.071484 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071374 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-node-log\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.071484 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a8cfb746-8899-4809-b938-560d777b6c85-konnectivity-ca\") pod \"konnectivity-agent-g4hc4\" (UID: \"a8cfb746-8899-4809-b938-560d777b6c85\") " pod="kube-system/konnectivity-agent-g4hc4" Sep 09 23:17:21.071484 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/410621c5-2a15-47eb-a4e4-842178ce68a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.071484 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071460 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5jbr\" (UniqueName: \"kubernetes.io/projected/887b5812-e7ca-4d43-a895-daa6e0005a8d-kube-api-access-k5jbr\") pod \"node-ca-v8n2v\" (UID: \"887b5812-e7ca-4d43-a895-daa6e0005a8d\") " pod="openshift-image-registry/node-ca-v8n2v" Sep 09 23:17:21.071484 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-kubelet\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.071812 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071534 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:21.071812 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071563 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-kubelet\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.071812 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071607 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-device-dir\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.071812 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.071685 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:21.071812 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071724 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-etc-selinux\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.071812 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071723 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 23:17:21.071812 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.071758 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs podName:63dd3bbf-4802-4b4a-9b06-37d9cf22ee94 nodeName:}" failed. No retries permitted until 2026-09-09 23:17:21.571736568 +0000 UTC m=+2.998159857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs") pod "network-metrics-daemon-vdx58" (UID: "63dd3bbf-4802-4b4a-9b06-37d9cf22ee94") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:21.071812 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071778 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/410621c5-2a15-47eb-a4e4-842178ce68a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071833 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-var-lib-kubelet\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071869 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b22436d1-37ee-45c9-af11-3ee766cb8e70-hosts-file\") pod \"node-resolver-fj28m\" (UID: \"b22436d1-37ee-45c9-af11-3ee766cb8e70\") " pod="openshift-dns/node-resolver-fj28m" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071890 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-socket-dir\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdrhv\" (UniqueName: \"kubernetes.io/projected/c2102a48-7202-4287-81c6-c3a6866ddd11-kube-api-access-wdrhv\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b326930-e182-4769-b41a-f79a91a2a5db-dbus\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.071985 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b22436d1-37ee-45c9-af11-3ee766cb8e70-hosts-file\") pod \"node-resolver-fj28m\" (UID: \"b22436d1-37ee-45c9-af11-3ee766cb8e70\") " pod="openshift-dns/node-resolver-fj28m" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072012 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6732b2ff-3cde-46f8-a772-8906dbf5a69b-iptables-alerter-script\") pod \"iptables-alerter-v6gvv\" (UID: \"6732b2ff-3cde-46f8-a772-8906dbf5a69b\") " pod="openshift-network-operator/iptables-alerter-v6gvv" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072042 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3c95a6dd-eb28-45c4-a536-b82de1e74698-multus-daemon-config\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072053 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a8cfb746-8899-4809-b938-560d777b6c85-konnectivity-ca\") pod \"konnectivity-agent-g4hc4\" (UID: \"a8cfb746-8899-4809-b938-560d777b6c85\") " pod="kube-system/konnectivity-agent-g4hc4" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072072 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b326930-e182-4769-b41a-f79a91a2a5db-dbus\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072078 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-kubernetes\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072109 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-run-netns\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-var-lib-openvswitch\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072158 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-kubernetes\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072156 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-log-socket\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072205 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-var-lib-openvswitch\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.072263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072210 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3132de00-554c-4204-b3f8-446e254f72e7-ovnkube-config\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072260 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-log-socket\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3132de00-554c-4204-b3f8-446e254f72e7-ovn-node-metrics-cert\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072205 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-run-netns\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd57l\" (UniqueName: \"kubernetes.io/projected/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-kube-api-access-nd57l\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072364 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rfp4\" (UniqueName: \"kubernetes.io/projected/410621c5-2a15-47eb-a4e4-842178ce68a7-kube-api-access-9rfp4\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-sysconfig\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072549 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2102a48-7202-4287-81c6-c3a6866ddd11-tmp\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072564 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6732b2ff-3cde-46f8-a772-8906dbf5a69b-iptables-alerter-script\") pod \"iptables-alerter-v6gvv\" (UID: \"6732b2ff-3cde-46f8-a772-8906dbf5a69b\") " pod="openshift-network-operator/iptables-alerter-v6gvv" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072597 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072641 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpxdz\" (UniqueName: \"kubernetes.io/projected/b22436d1-37ee-45c9-af11-3ee766cb8e70-kube-api-access-bpxdz\") pod \"node-resolver-fj28m\" (UID: \"b22436d1-37ee-45c9-af11-3ee766cb8e70\") " pod="openshift-dns/node-resolver-fj28m" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072627 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-sysconfig\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072671 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-system-cni-dir\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072711 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-multus-cni-dir\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072728 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3132de00-554c-4204-b3f8-446e254f72e7-ovnkube-config\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.072744 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072736 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-run-k8s-cni-cncf-io\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.073061 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.072794 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret podName:6b326930-e182-4769-b41a-f79a91a2a5db nodeName:}" failed. No retries permitted until 2026-09-09 23:17:21.572778055 +0000 UTC m=+2.999201354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret") pod "global-pull-secret-syncer-xxk7v" (UID: "6b326930-e182-4769-b41a-f79a91a2a5db") : object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072793 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-run-netns\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072839 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-systemd\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072868 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072892 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/410621c5-2a15-47eb-a4e4-842178ce68a7-system-cni-dir\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072900 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-systemd\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072916 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/410621c5-2a15-47eb-a4e4-842178ce68a7-cnibin\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072935 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/887b5812-e7ca-4d43-a895-daa6e0005a8d-host\") pod \"node-ca-v8n2v\" (UID: \"887b5812-e7ca-4d43-a895-daa6e0005a8d\") " pod="openshift-image-registry/node-ca-v8n2v" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.072957 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-multus-socket-dir-parent\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073054 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-var-lib-cni-multus\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-host\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073156 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-cni-bin\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073183 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-host\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6732b2ff-3cde-46f8-a772-8906dbf5a69b-host-slash\") pod \"iptables-alerter-v6gvv\" (UID: \"6732b2ff-3cde-46f8-a772-8906dbf5a69b\") " pod="openshift-network-operator/iptables-alerter-v6gvv" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073226 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c95a6dd-eb28-45c4-a536-b82de1e74698-cni-binary-copy\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073237 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-cni-bin\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073269 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6732b2ff-3cde-46f8-a772-8906dbf5a69b-host-slash\") pod \"iptables-alerter-v6gvv\" (UID: \"6732b2ff-3cde-46f8-a772-8906dbf5a69b\") " pod="openshift-network-operator/iptables-alerter-v6gvv" Sep 09 23:17:21.073881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073277 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-var-lib-cni-bin\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073317 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-run\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-lib-modules\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073400 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b326930-e182-4769-b41a-f79a91a2a5db-kubelet-config\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073423 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-run-systemd\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073372 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-run\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073442 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-lib-modules\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-etc-openvswitch\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073499 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-run-systemd\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-sysctl-d\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073524 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b326930-e182-4769-b41a-f79a91a2a5db-kubelet-config\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073534 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-etc-openvswitch\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073573 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-run-ovn\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073608 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073656 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3132de00-554c-4204-b3f8-446e254f72e7-ovnkube-script-lib\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073674 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-run-ovn\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.074724 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073714 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/410621c5-2a15-47eb-a4e4-842178ce68a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073750 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-sysctl-d\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073825 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-run-multus-certs\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073872 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-modprobe-d\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073911 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-slash\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073956 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-modprobe-d\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.073947 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074011 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-slash\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074060 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074098 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mq28\" (UniqueName: \"kubernetes.io/projected/50b17834-aa80-44f9-9951-7d429d662866-kube-api-access-5mq28\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3132de00-554c-4204-b3f8-446e254f72e7-env-overrides\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074177 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-registration-dir\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074202 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5hf2\" (UniqueName: \"kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2\") pod \"network-check-target-mr82q\" (UID: \"fbf8f0ca-22cf-4c56-89df-8a94b1ac7795\") " pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-multus-conf-dir\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074271 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-var-lib-kubelet\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pstmq\" (UniqueName: \"kubernetes.io/projected/3132de00-554c-4204-b3f8-446e254f72e7-kube-api-access-pstmq\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.075580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074350 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-sys-fs\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/410621c5-2a15-47eb-a4e4-842178ce68a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-sysctl-conf\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b22436d1-37ee-45c9-af11-3ee766cb8e70-tmp-dir\") pod \"node-resolver-fj28m\" (UID: \"b22436d1-37ee-45c9-af11-3ee766cb8e70\") " pod="openshift-dns/node-resolver-fj28m" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-systemd-units\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074500 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-cnibin\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074536 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-etc-kubernetes\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074561 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-sys\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-run-openvswitch\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074614 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-cni-netd\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074651 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3132de00-554c-4204-b3f8-446e254f72e7-env-overrides\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gxf\" (UniqueName: \"kubernetes.io/projected/6732b2ff-3cde-46f8-a772-8906dbf5a69b-kube-api-access-v7gxf\") pod \"iptables-alerter-v6gvv\" (UID: \"6732b2ff-3cde-46f8-a772-8906dbf5a69b\") " pod="openshift-network-operator/iptables-alerter-v6gvv" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074683 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/887b5812-e7ca-4d43-a895-daa6e0005a8d-serviceca\") pod \"node-ca-v8n2v\" (UID: \"887b5812-e7ca-4d43-a895-daa6e0005a8d\") " pod="openshift-image-registry/node-ca-v8n2v" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074707 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrx56\" (UniqueName: \"kubernetes.io/projected/3c95a6dd-eb28-45c4-a536-b82de1e74698-kube-api-access-xrx56\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074774 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-sysctl-conf\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074882 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-systemd-units\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.074959 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-sys\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.076283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.075021 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-run-openvswitch\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.076855 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.075086 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3132de00-554c-4204-b3f8-446e254f72e7-host-cni-netd\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.076855 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.075104 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b22436d1-37ee-45c9-af11-3ee766cb8e70-tmp-dir\") pod \"node-resolver-fj28m\" (UID: \"b22436d1-37ee-45c9-af11-3ee766cb8e70\") " pod="openshift-dns/node-resolver-fj28m" Sep 09 23:17:21.076855 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.075143 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2102a48-7202-4287-81c6-c3a6866ddd11-var-lib-kubelet\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.076855 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.075805 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3132de00-554c-4204-b3f8-446e254f72e7-ovnkube-script-lib\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.076855 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.076026 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2102a48-7202-4287-81c6-c3a6866ddd11-tmp\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.076855 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.076042 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c2102a48-7202-4287-81c6-c3a6866ddd11-etc-tuned\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.076855 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.076135 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3132de00-554c-4204-b3f8-446e254f72e7-ovn-node-metrics-cert\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.085355 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.085317 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a8cfb746-8899-4809-b938-560d777b6c85-agent-certs\") pod \"konnectivity-agent-g4hc4\" (UID: \"a8cfb746-8899-4809-b938-560d777b6c85\") " pod="kube-system/konnectivity-agent-g4hc4" Sep 09 23:17:21.085506 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.085405 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd57l\" (UniqueName: \"kubernetes.io/projected/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-kube-api-access-nd57l\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:21.085852 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.085809 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdrhv\" (UniqueName: \"kubernetes.io/projected/c2102a48-7202-4287-81c6-c3a6866ddd11-kube-api-access-wdrhv\") pod \"tuned-7w29b\" (UID: \"c2102a48-7202-4287-81c6-c3a6866ddd11\") " pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.086370 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.086330 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pstmq\" (UniqueName: \"kubernetes.io/projected/3132de00-554c-4204-b3f8-446e254f72e7-kube-api-access-pstmq\") pod \"ovnkube-node-66lv5\" (UID: \"3132de00-554c-4204-b3f8-446e254f72e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.086899 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.086877 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpxdz\" (UniqueName: \"kubernetes.io/projected/b22436d1-37ee-45c9-af11-3ee766cb8e70-kube-api-access-bpxdz\") pod \"node-resolver-fj28m\" (UID: \"b22436d1-37ee-45c9-af11-3ee766cb8e70\") " pod="openshift-dns/node-resolver-fj28m" Sep 09 23:17:21.087230 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.087207 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7gxf\" (UniqueName: \"kubernetes.io/projected/6732b2ff-3cde-46f8-a772-8906dbf5a69b-kube-api-access-v7gxf\") pod \"iptables-alerter-v6gvv\" (UID: \"6732b2ff-3cde-46f8-a772-8906dbf5a69b\") " pod="openshift-network-operator/iptables-alerter-v6gvv" Sep 09 23:17:21.175357 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3c95a6dd-eb28-45c4-a536-b82de1e74698-multus-daemon-config\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rfp4\" (UniqueName: \"kubernetes.io/projected/410621c5-2a15-47eb-a4e4-842178ce68a7-kube-api-access-9rfp4\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.175544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-system-cni-dir\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-multus-cni-dir\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175457 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-run-k8s-cni-cncf-io\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175481 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-run-netns\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175506 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.175544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/410621c5-2a15-47eb-a4e4-842178ce68a7-system-cni-dir\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.175544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175531 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-system-cni-dir\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175548 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-multus-cni-dir\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175599 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-run-netns\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175553 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/410621c5-2a15-47eb-a4e4-842178ce68a7-cnibin\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175616 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175550 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-run-k8s-cni-cncf-io\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175603 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/410621c5-2a15-47eb-a4e4-842178ce68a7-system-cni-dir\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/887b5812-e7ca-4d43-a895-daa6e0005a8d-host\") pod \"node-ca-v8n2v\" (UID: \"887b5812-e7ca-4d43-a895-daa6e0005a8d\") " pod="openshift-image-registry/node-ca-v8n2v" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175602 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/410621c5-2a15-47eb-a4e4-842178ce68a7-cnibin\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-multus-socket-dir-parent\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175687 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-var-lib-cni-multus\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175715 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c95a6dd-eb28-45c4-a536-b82de1e74698-cni-binary-copy\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175723 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-multus-socket-dir-parent\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175688 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/887b5812-e7ca-4d43-a895-daa6e0005a8d-host\") pod \"node-ca-v8n2v\" (UID: \"887b5812-e7ca-4d43-a895-daa6e0005a8d\") " pod="openshift-image-registry/node-ca-v8n2v" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175740 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-var-lib-cni-bin\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175776 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-var-lib-cni-bin\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175783 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/410621c5-2a15-47eb-a4e4-842178ce68a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175794 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-var-lib-cni-multus\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.175903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175812 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-run-multus-certs\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175842 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mq28\" (UniqueName: \"kubernetes.io/projected/50b17834-aa80-44f9-9951-7d429d662866-kube-api-access-5mq28\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175848 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-run-multus-certs\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175867 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-registration-dir\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175893 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5hf2\" (UniqueName: \"kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2\") pod \"network-check-target-mr82q\" (UID: \"fbf8f0ca-22cf-4c56-89df-8a94b1ac7795\") " pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-multus-conf-dir\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175942 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-sys-fs\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/410621c5-2a15-47eb-a4e4-842178ce68a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.175995 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3c95a6dd-eb28-45c4-a536-b82de1e74698-multus-daemon-config\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176009 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-cnibin\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176041 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-etc-kubernetes\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176064 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-cnibin\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176070 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/887b5812-e7ca-4d43-a895-daa6e0005a8d-serviceca\") pod \"node-ca-v8n2v\" (UID: \"887b5812-e7ca-4d43-a895-daa6e0005a8d\") " pod="openshift-image-registry/node-ca-v8n2v" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrx56\" (UniqueName: \"kubernetes.io/projected/3c95a6dd-eb28-45c4-a536-b82de1e74698-kube-api-access-xrx56\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176123 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-hostroot\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176146 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-sys-fs\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176149 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/410621c5-2a15-47eb-a4e4-842178ce68a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.176652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176183 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5jbr\" (UniqueName: \"kubernetes.io/projected/887b5812-e7ca-4d43-a895-daa6e0005a8d-kube-api-access-k5jbr\") pod \"node-ca-v8n2v\" (UID: \"887b5812-e7ca-4d43-a895-daa6e0005a8d\") " pod="openshift-image-registry/node-ca-v8n2v" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176222 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-device-dir\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-etc-selinux\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176256 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c95a6dd-eb28-45c4-a536-b82de1e74698-cni-binary-copy\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176267 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/410621c5-2a15-47eb-a4e4-842178ce68a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176287 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/410621c5-2a15-47eb-a4e4-842178ce68a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176293 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-var-lib-kubelet\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176329 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-socket-dir\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176349 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-etc-selinux\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176390 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-etc-kubernetes\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176429 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/410621c5-2a15-47eb-a4e4-842178ce68a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176436 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-registration-dir\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176450 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-socket-dir\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176569 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/410621c5-2a15-47eb-a4e4-842178ce68a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176610 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-multus-conf-dir\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176650 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/50b17834-aa80-44f9-9951-7d429d662866-device-dir\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176659 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-host-var-lib-kubelet\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.177260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.176761 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3c95a6dd-eb28-45c4-a536-b82de1e74698-hostroot\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.177813 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.177000 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/410621c5-2a15-47eb-a4e4-842178ce68a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.177813 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.177067 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/887b5812-e7ca-4d43-a895-daa6e0005a8d-serviceca\") pod \"node-ca-v8n2v\" (UID: \"887b5812-e7ca-4d43-a895-daa6e0005a8d\") " pod="openshift-image-registry/node-ca-v8n2v" Sep 09 23:17:21.182318 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.182293 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 23:17:21.182318 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.182322 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 23:17:21.182483 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.182334 2580 projected.go:196] Error preparing data for projected volume kube-api-access-l5hf2 for pod openshift-network-diagnostics/network-check-target-mr82q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:21.182483 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.182414 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2 podName:fbf8f0ca-22cf-4c56-89df-8a94b1ac7795 nodeName:}" failed. No retries permitted until 2026-09-09 23:17:21.682399526 +0000 UTC m=+3.108822830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l5hf2" (UniqueName: "kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2") pod "network-check-target-mr82q" (UID: "fbf8f0ca-22cf-4c56-89df-8a94b1ac7795") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:21.184901 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.184875 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rfp4\" (UniqueName: \"kubernetes.io/projected/410621c5-2a15-47eb-a4e4-842178ce68a7-kube-api-access-9rfp4\") pod \"multus-additional-cni-plugins-fbstm\" (UID: \"410621c5-2a15-47eb-a4e4-842178ce68a7\") " pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.184901 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.184886 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5jbr\" (UniqueName: \"kubernetes.io/projected/887b5812-e7ca-4d43-a895-daa6e0005a8d-kube-api-access-k5jbr\") pod \"node-ca-v8n2v\" (UID: \"887b5812-e7ca-4d43-a895-daa6e0005a8d\") " pod="openshift-image-registry/node-ca-v8n2v" Sep 09 23:17:21.185084 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.185037 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mq28\" (UniqueName: \"kubernetes.io/projected/50b17834-aa80-44f9-9951-7d429d662866-kube-api-access-5mq28\") pod \"aws-ebs-csi-driver-node-9shcn\" (UID: \"50b17834-aa80-44f9-9951-7d429d662866\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.186246 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.186225 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrx56\" (UniqueName: \"kubernetes.io/projected/3c95a6dd-eb28-45c4-a536-b82de1e74698-kube-api-access-xrx56\") pod \"multus-nw6cg\" (UID: \"3c95a6dd-eb28-45c4-a536-b82de1e74698\") " pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.254801 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.254765 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fj28m" Sep 09 23:17:21.262828 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.262809 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:21.274171 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.274142 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7w29b" Sep 09 23:17:21.282796 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.282764 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v6gvv" Sep 09 23:17:21.289818 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.289788 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g4hc4" Sep 09 23:17:21.297012 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.296989 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v8n2v" Sep 09 23:17:21.303098 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.303077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" Sep 09 23:17:21.310960 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.310937 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fbstm" Sep 09 23:17:21.316396 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.316372 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nw6cg" Sep 09 23:17:21.397983 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.397941 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 23:17:21.578997 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.578894 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:21.578997 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.578938 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:21.579193 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.579051 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:21.579193 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.579057 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:21.579193 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.579104 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret podName:6b326930-e182-4769-b41a-f79a91a2a5db nodeName:}" failed. No retries permitted until 2026-09-09 23:17:22.579090102 +0000 UTC m=+4.005513386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret") pod "global-pull-secret-syncer-xxk7v" (UID: "6b326930-e182-4769-b41a-f79a91a2a5db") : object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:21.579193 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.579117 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs podName:63dd3bbf-4802-4b4a-9b06-37d9cf22ee94 nodeName:}" failed. No retries permitted until 2026-09-09 23:17:22.579111277 +0000 UTC m=+4.005534562 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs") pod "network-metrics-daemon-vdx58" (UID: "63dd3bbf-4802-4b4a-9b06-37d9cf22ee94") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:21.780047 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.780017 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5hf2\" (UniqueName: \"kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2\") pod \"network-check-target-mr82q\" (UID: \"fbf8f0ca-22cf-4c56-89df-8a94b1ac7795\") " pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:21.780243 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.780222 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 23:17:21.780290 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.780255 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 23:17:21.780290 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.780269 2580 projected.go:196] Error preparing data for projected volume kube-api-access-l5hf2 for pod openshift-network-diagnostics/network-check-target-mr82q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:21.780386 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:21.780334 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2 podName:fbf8f0ca-22cf-4c56-89df-8a94b1ac7795 nodeName:}" failed. No retries permitted until 2026-09-09 23:17:22.780315146 +0000 UTC m=+4.206738438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l5hf2" (UniqueName: "kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2") pod "network-check-target-mr82q" (UID: "fbf8f0ca-22cf-4c56-89df-8a94b1ac7795") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:21.938911 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:21.938884 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c95a6dd_eb28_45c4_a536_b82de1e74698.slice/crio-07b65045ffc37361d17ca8231fc50a5e61cd3a40d33f610b3d6259c67104348f WatchSource:0}: Error finding container 07b65045ffc37361d17ca8231fc50a5e61cd3a40d33f610b3d6259c67104348f: Status 404 returned error can't find the container with id 07b65045ffc37361d17ca8231fc50a5e61cd3a40d33f610b3d6259c67104348f Sep 09 23:17:21.940394 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:21.940369 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3132de00_554c_4204_b3f8_446e254f72e7.slice/crio-686d7b14e855f4fef0d23b2f35ad8b587673d2a96e90035fbe051843347f24a7 WatchSource:0}: Error finding container 686d7b14e855f4fef0d23b2f35ad8b587673d2a96e90035fbe051843347f24a7: Status 404 returned error can't find the container with id 686d7b14e855f4fef0d23b2f35ad8b587673d2a96e90035fbe051843347f24a7 Sep 09 23:17:21.941725 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:21.941696 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50b17834_aa80_44f9_9951_7d429d662866.slice/crio-fd3a81d64b5fbfaad7bbb494034a7394445454f02406adc62dff8e70c03a865a WatchSource:0}: Error finding container fd3a81d64b5fbfaad7bbb494034a7394445454f02406adc62dff8e70c03a865a: Status 404 returned error can't find the container with id fd3a81d64b5fbfaad7bbb494034a7394445454f02406adc62dff8e70c03a865a Sep 09 23:17:21.945474 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:21.945321 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2102a48_7202_4287_81c6_c3a6866ddd11.slice/crio-32215cda1785bad47462f43caeb7b44ca6caff4310b32a6d98c33053e490d6f3 WatchSource:0}: Error finding container 32215cda1785bad47462f43caeb7b44ca6caff4310b32a6d98c33053e490d6f3: Status 404 returned error can't find the container with id 32215cda1785bad47462f43caeb7b44ca6caff4310b32a6d98c33053e490d6f3 Sep 09 23:17:21.947139 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:21.947030 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod887b5812_e7ca_4d43_a895_daa6e0005a8d.slice/crio-2573f892bf08375d8679a12e28f9b9065654f299ee2b5b05e4001a12f85e2843 WatchSource:0}: Error finding container 2573f892bf08375d8679a12e28f9b9065654f299ee2b5b05e4001a12f85e2843: Status 404 returned error can't find the container with id 2573f892bf08375d8679a12e28f9b9065654f299ee2b5b05e4001a12f85e2843 Sep 09 23:17:21.947672 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:21.947643 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6732b2ff_3cde_46f8_a772_8906dbf5a69b.slice/crio-1d21bc86c889638bb60b53be19144a4b13b4cfa95ce88402f7d1fc9831ac7b74 WatchSource:0}: Error finding container 1d21bc86c889638bb60b53be19144a4b13b4cfa95ce88402f7d1fc9831ac7b74: Status 404 returned error can't find the container with id 1d21bc86c889638bb60b53be19144a4b13b4cfa95ce88402f7d1fc9831ac7b74 Sep 09 23:17:21.948777 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:21.948753 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod410621c5_2a15_47eb_a4e4_842178ce68a7.slice/crio-d6646a003da35aa4426f413ffe08d6a7a60fe85e6f5b26eb162f64627445b810 WatchSource:0}: Error finding container d6646a003da35aa4426f413ffe08d6a7a60fe85e6f5b26eb162f64627445b810: Status 404 returned error can't find the container with id d6646a003da35aa4426f413ffe08d6a7a60fe85e6f5b26eb162f64627445b810 Sep 09 23:17:21.949629 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:21.949519 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb22436d1_37ee_45c9_af11_3ee766cb8e70.slice/crio-6f23c200b1ffd178af1abbab7bc41be7c4574c9580660a6b394642da4a624b96 WatchSource:0}: Error finding container 6f23c200b1ffd178af1abbab7bc41be7c4574c9580660a6b394642da4a624b96: Status 404 returned error can't find the container with id 6f23c200b1ffd178af1abbab7bc41be7c4574c9580660a6b394642da4a624b96 Sep 09 23:17:21.951417 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:17:21.950714 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8cfb746_8899_4809_b938_560d777b6c85.slice/crio-7d5bb6f79f17383b62295b48c8ef5acf8de429f26fd1964539d6e1ed0582b42f WatchSource:0}: Error finding container 7d5bb6f79f17383b62295b48c8ef5acf8de429f26fd1964539d6e1ed0582b42f: Status 404 returned error can't find the container with id 7d5bb6f79f17383b62295b48c8ef5acf8de429f26fd1964539d6e1ed0582b42f Sep 09 23:17:21.999259 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.999033 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 23:12:20 +0000 UTC" deadline="2028-03-14 18:32:26.292607275 +0000 UTC" Sep 09 23:17:21.999407 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:21.999261 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13243h15m4.293354552s" Sep 09 23:17:22.118014 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.117961 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:22.118154 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:22.118085 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:22.127485 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.127451 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-56.ec2.internal" event={"ID":"04642d3ff3a391e94be23b8faa39fe5f","Type":"ContainerStarted","Data":"9a6e5386a4e0ab54b6d52dc02b7fe82d718a5d322ac9e271a2c3bbd155b38166"} Sep 09 23:17:22.128642 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.128615 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fj28m" event={"ID":"b22436d1-37ee-45c9-af11-3ee766cb8e70","Type":"ContainerStarted","Data":"6f23c200b1ffd178af1abbab7bc41be7c4574c9580660a6b394642da4a624b96"} Sep 09 23:17:22.129794 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.129760 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v8n2v" event={"ID":"887b5812-e7ca-4d43-a895-daa6e0005a8d","Type":"ContainerStarted","Data":"2573f892bf08375d8679a12e28f9b9065654f299ee2b5b05e4001a12f85e2843"} Sep 09 23:17:22.131272 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.131235 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7w29b" event={"ID":"c2102a48-7202-4287-81c6-c3a6866ddd11","Type":"ContainerStarted","Data":"32215cda1785bad47462f43caeb7b44ca6caff4310b32a6d98c33053e490d6f3"} Sep 09 23:17:22.132258 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.132238 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" event={"ID":"50b17834-aa80-44f9-9951-7d429d662866","Type":"ContainerStarted","Data":"fd3a81d64b5fbfaad7bbb494034a7394445454f02406adc62dff8e70c03a865a"} Sep 09 23:17:22.133372 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.133342 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g4hc4" event={"ID":"a8cfb746-8899-4809-b938-560d777b6c85","Type":"ContainerStarted","Data":"7d5bb6f79f17383b62295b48c8ef5acf8de429f26fd1964539d6e1ed0582b42f"} Sep 09 23:17:22.134549 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.134524 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbstm" event={"ID":"410621c5-2a15-47eb-a4e4-842178ce68a7","Type":"ContainerStarted","Data":"d6646a003da35aa4426f413ffe08d6a7a60fe85e6f5b26eb162f64627445b810"} Sep 09 23:17:22.135727 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.135707 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v6gvv" event={"ID":"6732b2ff-3cde-46f8-a772-8906dbf5a69b","Type":"ContainerStarted","Data":"1d21bc86c889638bb60b53be19144a4b13b4cfa95ce88402f7d1fc9831ac7b74"} Sep 09 23:17:22.137046 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.137018 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" event={"ID":"3132de00-554c-4204-b3f8-446e254f72e7","Type":"ContainerStarted","Data":"686d7b14e855f4fef0d23b2f35ad8b587673d2a96e90035fbe051843347f24a7"} Sep 09 23:17:22.138303 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.138279 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nw6cg" event={"ID":"3c95a6dd-eb28-45c4-a536-b82de1e74698","Type":"ContainerStarted","Data":"07b65045ffc37361d17ca8231fc50a5e61cd3a40d33f610b3d6259c67104348f"} Sep 09 23:17:22.142359 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.142311 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-56.ec2.internal" podStartSLOduration=2.142299177 podStartE2EDuration="2.142299177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:17:22.141893161 +0000 UTC m=+3.568316468" watchObservedRunningTime="2026-09-09 23:17:22.142299177 +0000 UTC m=+3.568722484" Sep 09 23:17:22.585630 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.585546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:22.585630 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.585605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:22.585844 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:22.585753 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:22.585844 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:22.585814 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret podName:6b326930-e182-4769-b41a-f79a91a2a5db nodeName:}" failed. No retries permitted until 2026-09-09 23:17:24.585795645 +0000 UTC m=+6.012218953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret") pod "global-pull-secret-syncer-xxk7v" (UID: "6b326930-e182-4769-b41a-f79a91a2a5db") : object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:22.586253 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:22.586228 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:22.586376 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:22.586284 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs podName:63dd3bbf-4802-4b4a-9b06-37d9cf22ee94 nodeName:}" failed. No retries permitted until 2026-09-09 23:17:24.586270249 +0000 UTC m=+6.012693537 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs") pod "network-metrics-daemon-vdx58" (UID: "63dd3bbf-4802-4b4a-9b06-37d9cf22ee94") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:22.788921 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:22.786836 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5hf2\" (UniqueName: \"kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2\") pod \"network-check-target-mr82q\" (UID: \"fbf8f0ca-22cf-4c56-89df-8a94b1ac7795\") " pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:22.788921 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:22.787027 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 23:17:22.788921 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:22.787048 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 23:17:22.788921 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:22.787061 2580 projected.go:196] Error preparing data for projected volume kube-api-access-l5hf2 for pod openshift-network-diagnostics/network-check-target-mr82q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:22.788921 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:22.787120 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2 podName:fbf8f0ca-22cf-4c56-89df-8a94b1ac7795 nodeName:}" failed. No retries permitted until 2026-09-09 23:17:24.787101341 +0000 UTC m=+6.213524641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l5hf2" (UniqueName: "kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2") pod "network-check-target-mr82q" (UID: "fbf8f0ca-22cf-4c56-89df-8a94b1ac7795") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:23.000320 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:23.000181 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 23:12:20 +0000 UTC" deadline="2028-05-12 16:55:01.392808703 +0000 UTC" Sep 09 23:17:23.000320 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:23.000240 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14657h37m38.392572553s" Sep 09 23:17:23.118420 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:23.118362 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:23.118599 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:23.118502 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:23.120236 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:23.120084 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:23.120236 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:23.120195 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:23.151640 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:23.151591 2580 generic.go:356] "Generic (PLEG): container finished" podID="6bb598d832714d83102c4c0968944793" containerID="1b60ccf5821a8118512df68cb299214eadf532f891a25b83daf8639fb7b834c9" exitCode=0 Sep 09 23:17:23.152577 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:23.152544 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" event={"ID":"6bb598d832714d83102c4c0968944793","Type":"ContainerDied","Data":"1b60ccf5821a8118512df68cb299214eadf532f891a25b83daf8639fb7b834c9"} Sep 09 23:17:24.118806 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:24.118247 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:24.118806 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:24.118373 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:24.191007 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:24.190071 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" event={"ID":"6bb598d832714d83102c4c0968944793","Type":"ContainerStarted","Data":"b1d36029b93db0f225da71fd8683fa16878c5d77881132d877fd2c95876e7d9e"} Sep 09 23:17:24.205985 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:24.205917 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-56.ec2.internal" podStartSLOduration=4.205899074 podStartE2EDuration="4.205899074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:17:24.20544377 +0000 UTC m=+5.631867078" watchObservedRunningTime="2026-09-09 23:17:24.205899074 +0000 UTC m=+5.632322382" Sep 09 23:17:24.603079 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:24.602743 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:24.603079 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:24.602805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:24.603079 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:24.602938 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:24.603079 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:24.603018 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret podName:6b326930-e182-4769-b41a-f79a91a2a5db nodeName:}" failed. No retries permitted until 2026-09-09 23:17:28.60299894 +0000 UTC m=+10.029422248 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret") pod "global-pull-secret-syncer-xxk7v" (UID: "6b326930-e182-4769-b41a-f79a91a2a5db") : object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:24.603599 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:24.603414 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:24.603599 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:24.603466 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs podName:63dd3bbf-4802-4b4a-9b06-37d9cf22ee94 nodeName:}" failed. No retries permitted until 2026-09-09 23:17:28.603451624 +0000 UTC m=+10.029874924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs") pod "network-metrics-daemon-vdx58" (UID: "63dd3bbf-4802-4b4a-9b06-37d9cf22ee94") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:24.804297 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:24.804257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5hf2\" (UniqueName: \"kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2\") pod \"network-check-target-mr82q\" (UID: \"fbf8f0ca-22cf-4c56-89df-8a94b1ac7795\") " pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:24.804480 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:24.804464 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 23:17:24.804543 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:24.804494 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 23:17:24.804543 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:24.804509 2580 projected.go:196] Error preparing data for projected volume kube-api-access-l5hf2 for pod openshift-network-diagnostics/network-check-target-mr82q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:24.804640 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:24.804580 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2 podName:fbf8f0ca-22cf-4c56-89df-8a94b1ac7795 nodeName:}" failed. No retries permitted until 2026-09-09 23:17:28.804560901 +0000 UTC m=+10.230984200 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l5hf2" (UniqueName: "kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2") pod "network-check-target-mr82q" (UID: "fbf8f0ca-22cf-4c56-89df-8a94b1ac7795") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:25.119605 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:25.119562 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:25.120081 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:25.119696 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:25.120422 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:25.120212 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:25.120422 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:25.120363 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:26.117954 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:26.117908 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:26.118196 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:26.118061 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:27.118243 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:27.118207 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:27.118677 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:27.118207 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:27.118677 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:27.118364 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:27.118677 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:27.118417 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:28.118931 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:28.118467 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:28.118931 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:28.118590 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:28.634686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:28.634650 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:28.634827 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:28.634742 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:28.634827 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:28.634802 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:28.634827 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:28.634818 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:28.634935 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:28.634873 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs podName:63dd3bbf-4802-4b4a-9b06-37d9cf22ee94 nodeName:}" failed. No retries permitted until 2026-09-09 23:17:36.634854358 +0000 UTC m=+18.061277649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs") pod "network-metrics-daemon-vdx58" (UID: "63dd3bbf-4802-4b4a-9b06-37d9cf22ee94") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:28.634935 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:28.634891 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret podName:6b326930-e182-4769-b41a-f79a91a2a5db nodeName:}" failed. No retries permitted until 2026-09-09 23:17:36.634881485 +0000 UTC m=+18.061304784 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret") pod "global-pull-secret-syncer-xxk7v" (UID: "6b326930-e182-4769-b41a-f79a91a2a5db") : object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:28.836835 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:28.836136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5hf2\" (UniqueName: \"kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2\") pod \"network-check-target-mr82q\" (UID: \"fbf8f0ca-22cf-4c56-89df-8a94b1ac7795\") " pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:28.836835 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:28.836370 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 23:17:28.836835 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:28.836392 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 23:17:28.836835 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:28.836405 2580 projected.go:196] Error preparing data for projected volume kube-api-access-l5hf2 for pod openshift-network-diagnostics/network-check-target-mr82q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:28.836835 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:28.836467 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2 podName:fbf8f0ca-22cf-4c56-89df-8a94b1ac7795 nodeName:}" failed. No retries permitted until 2026-09-09 23:17:36.836445598 +0000 UTC m=+18.262868887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l5hf2" (UniqueName: "kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2") pod "network-check-target-mr82q" (UID: "fbf8f0ca-22cf-4c56-89df-8a94b1ac7795") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:29.119571 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:29.119202 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:29.119571 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:29.119314 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:29.119571 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:29.119395 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:29.119571 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:29.119485 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:30.119182 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:30.118703 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:30.119182 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:30.118827 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:31.118530 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:31.118497 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:31.118929 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:31.118630 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:31.118929 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:31.118710 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:31.118929 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:31.118829 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:32.118322 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:32.118293 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:32.118459 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:32.118389 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:33.118824 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:33.118789 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:33.119294 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:33.118924 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:33.119294 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:33.118992 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:33.119294 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:33.119117 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:34.118697 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:34.118667 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:34.118869 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:34.118776 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:35.121262 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:35.121233 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:35.121682 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:35.121368 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:35.121682 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:35.121408 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:35.121682 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:35.121528 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:36.119005 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:36.118802 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:36.119245 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:36.119097 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:36.697509 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:36.697470 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:36.697509 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:36.697521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:36.698072 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:36.697631 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:36.698072 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:36.697635 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:36.698072 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:36.697693 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret podName:6b326930-e182-4769-b41a-f79a91a2a5db nodeName:}" failed. No retries permitted until 2026-09-09 23:17:52.697678216 +0000 UTC m=+34.124101514 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret") pod "global-pull-secret-syncer-xxk7v" (UID: "6b326930-e182-4769-b41a-f79a91a2a5db") : object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:36.698072 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:36.697711 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs podName:63dd3bbf-4802-4b4a-9b06-37d9cf22ee94 nodeName:}" failed. No retries permitted until 2026-09-09 23:17:52.697703607 +0000 UTC m=+34.124126906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs") pod "network-metrics-daemon-vdx58" (UID: "63dd3bbf-4802-4b4a-9b06-37d9cf22ee94") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:36.899660 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:36.899631 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5hf2\" (UniqueName: \"kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2\") pod \"network-check-target-mr82q\" (UID: \"fbf8f0ca-22cf-4c56-89df-8a94b1ac7795\") " pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:36.899835 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:36.899762 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 23:17:36.899835 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:36.899782 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 23:17:36.899835 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:36.899790 2580 projected.go:196] Error preparing data for projected volume kube-api-access-l5hf2 for pod openshift-network-diagnostics/network-check-target-mr82q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:36.899990 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:36.899839 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2 podName:fbf8f0ca-22cf-4c56-89df-8a94b1ac7795 nodeName:}" failed. No retries permitted until 2026-09-09 23:17:52.899824653 +0000 UTC m=+34.326247958 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l5hf2" (UniqueName: "kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2") pod "network-check-target-mr82q" (UID: "fbf8f0ca-22cf-4c56-89df-8a94b1ac7795") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:37.118135 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:37.118055 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:37.118286 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:37.118191 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:37.118286 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:37.118252 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:37.118405 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:37.118372 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:38.118546 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:38.118515 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:38.119022 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:38.118638 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:39.118943 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:39.118908 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:39.119402 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:39.119007 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:39.119402 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:39.119029 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:39.119402 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:39.119134 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:40.118497 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.118317 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:40.118591 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:40.118572 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:40.221840 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.221812 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g4hc4" event={"ID":"a8cfb746-8899-4809-b938-560d777b6c85","Type":"ContainerStarted","Data":"d7ecc711b181c42ea6e9a8bf44ab77c7fadb36b90dff4a008ba2b31ed919a310"} Sep 09 23:17:40.223795 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.223771 2580 generic.go:356] "Generic (PLEG): container finished" podID="410621c5-2a15-47eb-a4e4-842178ce68a7" containerID="96d3f143f14aa3943754b05287d0180880b2bbaeb8fca49bc6878c87bb97579d" exitCode=0 Sep 09 23:17:40.223893 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.223863 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbstm" event={"ID":"410621c5-2a15-47eb-a4e4-842178ce68a7","Type":"ContainerDied","Data":"96d3f143f14aa3943754b05287d0180880b2bbaeb8fca49bc6878c87bb97579d"} Sep 09 23:17:40.226652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.226633 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/0.log" Sep 09 23:17:40.227037 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.227014 2580 generic.go:356] "Generic (PLEG): container finished" podID="3132de00-554c-4204-b3f8-446e254f72e7" containerID="abc218d8956ac60392878924edcd7bf5e1e83da023239bb0788d1787967f5876" exitCode=1 Sep 09 23:17:40.227122 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.227042 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" event={"ID":"3132de00-554c-4204-b3f8-446e254f72e7","Type":"ContainerStarted","Data":"c28d07c74057b5c58a2ee9cd0860ad5906d7eef4fdc565c1d9c8565df4d32b04"} Sep 09 23:17:40.227122 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.227099 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" event={"ID":"3132de00-554c-4204-b3f8-446e254f72e7","Type":"ContainerStarted","Data":"8d00bdd1519afc8c4b27202d42dbe3c6494bc19b65b3493d7f9e50f4f49a076f"} Sep 09 23:17:40.227290 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.227136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" event={"ID":"3132de00-554c-4204-b3f8-446e254f72e7","Type":"ContainerStarted","Data":"a49dd9c4e09b052cb7691cf5a77a7ed054244b4ff6a921ce8aa43b60b60a27e3"} Sep 09 23:17:40.227290 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.227179 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" event={"ID":"3132de00-554c-4204-b3f8-446e254f72e7","Type":"ContainerDied","Data":"abc218d8956ac60392878924edcd7bf5e1e83da023239bb0788d1787967f5876"} Sep 09 23:17:40.227290 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.227211 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" event={"ID":"3132de00-554c-4204-b3f8-446e254f72e7","Type":"ContainerStarted","Data":"0a0a906911b8aeabe63c7aec9d0714f3990c136d54aa6d0eb8e58768bf7d5704"} Sep 09 23:17:40.228667 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.228647 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nw6cg" event={"ID":"3c95a6dd-eb28-45c4-a536-b82de1e74698","Type":"ContainerStarted","Data":"d5aa4ab861011a642ba16835064ecd2e16c4dd8bacd4aadb074ab33c11a898e4"} Sep 09 23:17:40.230193 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.230172 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fj28m" event={"ID":"b22436d1-37ee-45c9-af11-3ee766cb8e70","Type":"ContainerStarted","Data":"174ceb1c88d1e0e94aaba59aa1828c9d440a5c3197d149d31aa415e2dc281705"} Sep 09 23:17:40.231635 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.231620 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v8n2v" event={"ID":"887b5812-e7ca-4d43-a895-daa6e0005a8d","Type":"ContainerStarted","Data":"5d5e4e117e5b3c3d9a26ac21cc741dd50fc8a1af21f556e5b869a65abdd7c91e"} Sep 09 23:17:40.233161 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.233136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7w29b" event={"ID":"c2102a48-7202-4287-81c6-c3a6866ddd11","Type":"ContainerStarted","Data":"55b531a43a3cefe8434f3cb8e14563e1c157bb06f1b3d7cda49cfa3a65ccfc64"} Sep 09 23:17:40.234502 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.234477 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" event={"ID":"50b17834-aa80-44f9-9951-7d429d662866","Type":"ContainerStarted","Data":"ae6e79e2d2faf3a635d6012d153261a93ed053004cf1f7ef5a52bcecd9a41eab"} Sep 09 23:17:40.237329 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.237285 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g4hc4" podStartSLOduration=3.854758894 podStartE2EDuration="21.237270412s" totalImagesPullingTime="17.382511518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:17:40.2370724 +0000 UTC m=+21.663495708" watchObservedRunningTime="2026-09-09 23:17:40.237270412 +0000 UTC m=+21.663693737" Sep 09 23:17:40.270450 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.270412 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v8n2v" podStartSLOduration=3.590941807 podStartE2EDuration="21.270399968s" totalImagesPullingTime="17.679458161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:17:40.270171305 +0000 UTC m=+21.696594612" watchObservedRunningTime="2026-09-09 23:17:40.270399968 +0000 UTC m=+21.696823272" Sep 09 23:17:40.287693 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.287644 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nw6cg" podStartSLOduration=3.317689059 podStartE2EDuration="21.287625954s" totalImagesPullingTime="17.969936895s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:17:40.287072046 +0000 UTC m=+21.713495353" watchObservedRunningTime="2026-09-09 23:17:40.287625954 +0000 UTC m=+21.714049263" Sep 09 23:17:40.300406 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.300362 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fj28m" podStartSLOduration=3.622029646 podStartE2EDuration="21.300351961s" totalImagesPullingTime="17.678322315s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:17:40.300320853 +0000 UTC m=+21.726744162" watchObservedRunningTime="2026-09-09 23:17:40.300351961 +0000 UTC m=+21.726775246" Sep 09 23:17:40.315899 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:40.315860 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7w29b" podStartSLOduration=3.633583968 podStartE2EDuration="21.315848221s" totalImagesPullingTime="17.682264253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:17:40.315342077 +0000 UTC m=+21.741765394" watchObservedRunningTime="2026-09-09 23:17:40.315848221 +0000 UTC m=+21.742271527" Sep 09 23:17:41.118563 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:41.118512 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:41.118697 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:41.118559 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:41.118697 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:41.118658 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:41.118808 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:41.118775 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:41.187280 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:41.187248 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 23:17:41.247452 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:41.247420 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" event={"ID":"50b17834-aa80-44f9-9951-7d429d662866","Type":"ContainerStarted","Data":"845f2416123a459b34768ed0b11e8f786bf6b39a700977aa2f0faba7a3b8b06e"} Sep 09 23:17:41.249201 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:41.249172 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v6gvv" event={"ID":"6732b2ff-3cde-46f8-a772-8906dbf5a69b","Type":"ContainerStarted","Data":"1a9d1f3ccccef78dde8019e114fb93e67e797782cdbed167f1f9285e21e903c6"} Sep 09 23:17:41.252250 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:41.252231 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/0.log" Sep 09 23:17:41.252774 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:41.252731 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" event={"ID":"3132de00-554c-4204-b3f8-446e254f72e7","Type":"ContainerStarted","Data":"cc7963258f2d6346840bd6bff70aa2d7602cc36715fc53eec8cae82730c41152"} Sep 09 23:17:41.263230 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:41.263183 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-v6gvv" podStartSLOduration=4.632755294 podStartE2EDuration="22.263164572s" totalImagesPullingTime="17.630409278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:17:41.262322982 +0000 UTC m=+22.688746291" watchObservedRunningTime="2026-09-09 23:17:41.263164572 +0000 UTC m=+22.689587871" Sep 09 23:17:42.033396 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:42.033311 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T23:17:41.187266519Z","UUID":"68b0619c-d16e-4fa0-ae37-3ce5667e9f89","Handler":null,"Name":"","Endpoint":""} Sep 09 23:17:42.034832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:42.034806 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 23:17:42.034948 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:42.034840 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 23:17:42.118246 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:42.118213 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:42.118508 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:42.118351 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:42.772888 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:42.772667 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g4hc4" Sep 09 23:17:42.773811 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:42.773791 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g4hc4" Sep 09 23:17:43.118152 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:43.118123 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:43.118152 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:43.118140 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:43.118410 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:43.118259 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:43.118410 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:43.118395 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:43.260922 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:43.260897 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/0.log" Sep 09 23:17:43.261361 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:43.261337 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" event={"ID":"3132de00-554c-4204-b3f8-446e254f72e7","Type":"ContainerStarted","Data":"1ab7ef3287beb1aa6982e19434035b9370a1cf2df51e3c8f78193e9fb6bcac38"} Sep 09 23:17:43.267372 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:43.267337 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" event={"ID":"50b17834-aa80-44f9-9951-7d429d662866","Type":"ContainerStarted","Data":"a8df275f43b2b724969b571c9512302a5f69a913285e999fee1d145ee23fca1d"} Sep 09 23:17:43.267503 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:43.267458 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g4hc4" Sep 09 23:17:43.267943 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:43.267925 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g4hc4" Sep 09 23:17:43.297216 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:43.297172 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shcn" podStartSLOduration=4.215980944 podStartE2EDuration="24.297160143s" totalImagesPullingTime="20.081179199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:17:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:17:43.296629079 +0000 UTC m=+24.723052387" watchObservedRunningTime="2026-09-09 23:17:43.297160143 +0000 UTC m=+24.723583451" Sep 09 23:17:44.118816 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:44.118784 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:44.119264 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:44.118906 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:45.118168 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:45.118063 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:45.118168 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:45.118074 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:45.118434 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:45.118170 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:45.118434 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:45.118309 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:45.275089 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:45.275019 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/0.log" Sep 09 23:17:45.275464 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:45.275433 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" event={"ID":"3132de00-554c-4204-b3f8-446e254f72e7","Type":"ContainerStarted","Data":"a19023184e46eede4b29f191747aca51688d790d263b056efd432b54cb00d216"} Sep 09 23:17:45.275793 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:45.275773 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:45.275877 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:45.275856 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:45.275925 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:45.275897 2580 scope.go:117] "RemoveContainer" containerID="abc218d8956ac60392878924edcd7bf5e1e83da023239bb0788d1787967f5876" Sep 09 23:17:45.275925 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:45.275911 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:45.291211 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:45.291183 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:45.291426 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:45.291406 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:17:46.118330 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:46.118141 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:46.118570 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:46.118431 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:46.279555 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:46.279476 2580 generic.go:356] "Generic (PLEG): container finished" podID="410621c5-2a15-47eb-a4e4-842178ce68a7" containerID="2015948ee32664ab8abeb75c302963771568e620904586af0928721deea6b243" exitCode=0 Sep 09 23:17:46.280007 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:46.279566 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbstm" event={"ID":"410621c5-2a15-47eb-a4e4-842178ce68a7","Type":"ContainerDied","Data":"2015948ee32664ab8abeb75c302963771568e620904586af0928721deea6b243"} Sep 09 23:17:46.283628 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:46.283611 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/0.log" Sep 09 23:17:46.284041 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:46.284019 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" event={"ID":"3132de00-554c-4204-b3f8-446e254f72e7","Type":"ContainerStarted","Data":"309d167f856a4ac641d070760637727d5829dea9814d422095d1b334b2698d69"} Sep 09 23:17:46.319400 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:46.319365 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" podStartSLOduration=9.59478098 podStartE2EDuration="27.319352978s" totalImagesPullingTime="17.724571998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:17:46.319219918 +0000 UTC m=+27.745643226" watchObservedRunningTime="2026-09-09 23:17:46.319352978 +0000 UTC m=+27.745776315" Sep 09 23:17:47.118073 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:47.118046 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:47.118213 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:47.118046 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:47.118213 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:47.118152 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:47.118298 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:47.118230 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:48.118333 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:48.118301 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:48.118833 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:48.118402 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:48.290413 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:48.290378 2580 generic.go:356] "Generic (PLEG): container finished" podID="410621c5-2a15-47eb-a4e4-842178ce68a7" containerID="18aef9b4c7d7da3d14d44efa92c6d8f6ad379c182b594693654cfd2603beda83" exitCode=0 Sep 09 23:17:48.290563 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:48.290424 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbstm" event={"ID":"410621c5-2a15-47eb-a4e4-842178ce68a7","Type":"ContainerDied","Data":"18aef9b4c7d7da3d14d44efa92c6d8f6ad379c182b594693654cfd2603beda83"} Sep 09 23:17:49.119537 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:49.119501 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:49.119995 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:49.119632 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:49.119995 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:49.119673 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:49.119995 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:49.119759 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:49.294670 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:49.294633 2580 generic.go:356] "Generic (PLEG): container finished" podID="410621c5-2a15-47eb-a4e4-842178ce68a7" containerID="d925324ac01e6eae31c5e468235d1f0a54e336af0276fb0f244d604f85214004" exitCode=0 Sep 09 23:17:49.294824 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:49.294709 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbstm" event={"ID":"410621c5-2a15-47eb-a4e4-842178ce68a7","Type":"ContainerDied","Data":"d925324ac01e6eae31c5e468235d1f0a54e336af0276fb0f244d604f85214004"} Sep 09 23:17:50.118075 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:50.118041 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:50.118264 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:50.118164 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:51.118835 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:51.118800 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:51.119277 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:51.118844 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:51.119277 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:51.118938 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:51.119277 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:51.119101 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:52.118808 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:52.118780 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:52.119060 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:52.118882 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:52.718471 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:52.718437 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:52.718664 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:52.718531 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:52.718664 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:52.718598 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:52.718664 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:52.718635 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:52.718822 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:52.718675 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret podName:6b326930-e182-4769-b41a-f79a91a2a5db nodeName:}" failed. No retries permitted until 2026-09-09 23:18:24.718654006 +0000 UTC m=+66.145077304 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret") pod "global-pull-secret-syncer-xxk7v" (UID: "6b326930-e182-4769-b41a-f79a91a2a5db") : object "kube-system"/"original-pull-secret" not registered Sep 09 23:17:52.718822 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:52.718696 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs podName:63dd3bbf-4802-4b4a-9b06-37d9cf22ee94 nodeName:}" failed. No retries permitted until 2026-09-09 23:18:24.718685899 +0000 UTC m=+66.145109184 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs") pod "network-metrics-daemon-vdx58" (UID: "63dd3bbf-4802-4b4a-9b06-37d9cf22ee94") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 23:17:52.920402 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:52.920359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5hf2\" (UniqueName: \"kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2\") pod \"network-check-target-mr82q\" (UID: \"fbf8f0ca-22cf-4c56-89df-8a94b1ac7795\") " pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:52.920572 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:52.920526 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 23:17:52.920572 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:52.920548 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 23:17:52.920572 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:52.920559 2580 projected.go:196] Error preparing data for projected volume kube-api-access-l5hf2 for pod openshift-network-diagnostics/network-check-target-mr82q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:52.920696 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:52.920624 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2 podName:fbf8f0ca-22cf-4c56-89df-8a94b1ac7795 nodeName:}" failed. No retries permitted until 2026-09-09 23:18:24.920604421 +0000 UTC m=+66.347027726 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-l5hf2" (UniqueName: "kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2") pod "network-check-target-mr82q" (UID: "fbf8f0ca-22cf-4c56-89df-8a94b1ac7795") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 23:17:53.118433 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:53.118404 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:53.118433 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:53.118405 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:53.118669 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:53.118549 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:53.118727 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:53.118705 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:54.118017 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:54.117991 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:54.118364 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:54.118120 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:54.307750 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:54.307724 2580 generic.go:356] "Generic (PLEG): container finished" podID="410621c5-2a15-47eb-a4e4-842178ce68a7" containerID="809e3ac00d2a605369625f676b42edc5cd044cd471ef00c69d45333bef040aa9" exitCode=0 Sep 09 23:17:54.307860 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:54.307787 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbstm" event={"ID":"410621c5-2a15-47eb-a4e4-842178ce68a7","Type":"ContainerDied","Data":"809e3ac00d2a605369625f676b42edc5cd044cd471ef00c69d45333bef040aa9"} Sep 09 23:17:55.118489 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:55.118457 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:55.118929 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:55.118589 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:55.118929 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:55.118634 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:55.118929 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:55.118730 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:55.313019 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:55.312988 2580 generic.go:356] "Generic (PLEG): container finished" podID="410621c5-2a15-47eb-a4e4-842178ce68a7" containerID="44babd333bbafd86559359c25219346a81032890552337739585e4b1a3fffbe3" exitCode=0 Sep 09 23:17:55.313196 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:55.313048 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbstm" event={"ID":"410621c5-2a15-47eb-a4e4-842178ce68a7","Type":"ContainerDied","Data":"44babd333bbafd86559359c25219346a81032890552337739585e4b1a3fffbe3"} Sep 09 23:17:56.118158 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:56.117962 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:56.118356 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:56.118230 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:56.320716 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:56.320670 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbstm" event={"ID":"410621c5-2a15-47eb-a4e4-842178ce68a7","Type":"ContainerStarted","Data":"f1e10fc8506647e00b6e54c5a8eab187d8329fbcf31d8916d4113939523b45a4"} Sep 09 23:17:56.340404 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:56.340345 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fbstm" podStartSLOduration=8.236774701 podStartE2EDuration="37.340325348s" totalImagesPullingTime="29.103550647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:17:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:17:56.339924139 +0000 UTC m=+37.766347448" watchObservedRunningTime="2026-09-09 23:17:56.340325348 +0000 UTC m=+37.766748657" Sep 09 23:17:57.066361 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:57.063396 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xxk7v"] Sep 09 23:17:57.066361 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:57.063767 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:57.066361 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:57.063931 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:17:57.066742 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:57.066722 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mr82q"] Sep 09 23:17:57.066882 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:57.066869 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vdx58"] Sep 09 23:17:57.067094 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:57.067080 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:57.067378 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:57.067334 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:57.067709 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:57.067689 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:57.068041 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:57.067937 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:59.119473 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:59.119440 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:17:59.119979 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:59.119529 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:17:59.119979 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:59.119617 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:17:59.119979 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:59.119727 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:17:59.119979 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:17:59.119792 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:17:59.119979 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:17:59.119857 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:18:01.118553 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.118516 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:18:01.118939 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.118521 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:18:01.118939 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:18:01.118635 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vdx58" podUID="63dd3bbf-4802-4b4a-9b06-37d9cf22ee94" Sep 09 23:18:01.118939 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.118526 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:18:01.118939 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:18:01.118708 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr82q" podUID="fbf8f0ca-22cf-4c56-89df-8a94b1ac7795" Sep 09 23:18:01.118939 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:18:01.118797 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxk7v" podUID="6b326930-e182-4769-b41a-f79a91a2a5db" Sep 09 23:18:01.834701 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.834672 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-56.ec2.internal" event="NodeReady" Sep 09 23:18:01.834868 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.834785 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 23:18:01.871795 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.871772 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b6f97fb9d-wzhqp"] Sep 09 23:18:01.893338 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.893310 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5rjt2"] Sep 09 23:18:01.893485 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.893429 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:01.896771 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.896749 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 09 23:18:01.896925 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.896776 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 09 23:18:01.897592 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.897539 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r67d2\"" Sep 09 23:18:01.898252 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.898230 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 09 23:18:01.903012 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.902859 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 09 23:18:01.908018 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.907998 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b6f97fb9d-wzhqp"] Sep 09 23:18:01.908098 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.908028 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kns46"] Sep 09 23:18:01.908098 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.908048 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:01.911043 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.911022 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 23:18:01.911276 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.911252 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w7vc5\"" Sep 09 23:18:01.911492 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.911479 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 23:18:01.926271 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.926249 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5rjt2"] Sep 09 23:18:01.926369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.926295 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kns46"] Sep 09 23:18:01.926369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.926317 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ztk45"] Sep 09 23:18:01.926369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.926358 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kns46" Sep 09 23:18:01.929591 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.928682 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 23:18:01.929591 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.929394 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wxhvm\"" Sep 09 23:18:01.929591 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.929394 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 23:18:01.929591 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.929483 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 23:18:01.948330 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.948305 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ztk45"] Sep 09 23:18:01.948427 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.948346 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pjndv"] Sep 09 23:18:01.948427 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.948412 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:01.951276 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.951255 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 23:18:01.951380 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.951308 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 23:18:01.951500 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.951479 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lnpcr\"" Sep 09 23:18:01.951552 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.951534 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 23:18:01.951607 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.951567 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 23:18:01.968467 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.968445 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pjndv"] Sep 09 23:18:01.968557 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.968537 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:01.971572 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.971554 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 23:18:01.971572 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.971564 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 09 23:18:01.971572 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.971554 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 23:18:01.971858 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.971594 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 09 23:18:01.971858 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.971655 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-66cd9\"" Sep 09 23:18:01.971858 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.971662 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 23:18:01.985533 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.985507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/567767c5-25a4-44b6-9f50-446d1ac6d3d6-registry-certificates\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:01.985630 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.985551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/567767c5-25a4-44b6-9f50-446d1ac6d3d6-image-registry-private-configuration\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:01.985630 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.985569 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/567767c5-25a4-44b6-9f50-446d1ac6d3d6-bound-sa-token\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:01.985630 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.985585 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/567767c5-25a4-44b6-9f50-446d1ac6d3d6-registry-tls\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:01.985747 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.985636 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/567767c5-25a4-44b6-9f50-446d1ac6d3d6-ca-trust-extracted\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:01.985747 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.985671 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvx9w\" (UniqueName: \"kubernetes.io/projected/567767c5-25a4-44b6-9f50-446d1ac6d3d6-kube-api-access-nvx9w\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:01.985747 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.985692 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/567767c5-25a4-44b6-9f50-446d1ac6d3d6-installation-pull-secrets\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:01.985747 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:01.985710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/567767c5-25a4-44b6-9f50-446d1ac6d3d6-trusted-ca\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.086206 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086110 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/567767c5-25a4-44b6-9f50-446d1ac6d3d6-image-registry-private-configuration\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.086206 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086147 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/567767c5-25a4-44b6-9f50-446d1ac6d3d6-bound-sa-token\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.086206 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086166 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a33577f-f30a-4592-90e3-2915e9bad561-config-volume\") pod \"dns-default-5rjt2\" (UID: \"5a33577f-f30a-4592-90e3-2915e9bad561\") " pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.086206 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086181 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b464eccc-dbbb-4c62-9697-a23d8bdb846e-data-volume\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.086206 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086204 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/567767c5-25a4-44b6-9f50-446d1ac6d3d6-registry-tls\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.086565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a33577f-f30a-4592-90e3-2915e9bad561-metrics-tls\") pod \"dns-default-5rjt2\" (UID: \"5a33577f-f30a-4592-90e3-2915e9bad561\") " pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.086565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086322 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/180bec08-a68e-4886-8f1f-126fafcafbc5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.086565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086345 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/180bec08-a68e-4886-8f1f-126fafcafbc5-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.086565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086371 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b464eccc-dbbb-4c62-9697-a23d8bdb846e-crio-socket\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.086565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086400 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/567767c5-25a4-44b6-9f50-446d1ac6d3d6-ca-trust-extracted\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.086565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086423 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twgdq\" (UniqueName: \"kubernetes.io/projected/180bec08-a68e-4886-8f1f-126fafcafbc5-kube-api-access-twgdq\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.086565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086438 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shq84\" (UniqueName: \"kubernetes.io/projected/5a33577f-f30a-4592-90e3-2915e9bad561-kube-api-access-shq84\") pod \"dns-default-5rjt2\" (UID: \"5a33577f-f30a-4592-90e3-2915e9bad561\") " pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.086565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086470 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58265cb5-d4cf-45b7-b827-7aa1717c810f-cert\") pod \"ingress-canary-kns46\" (UID: \"58265cb5-d4cf-45b7-b827-7aa1717c810f\") " pod="openshift-ingress-canary/ingress-canary-kns46" Sep 09 23:18:02.086565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9lrb\" (UniqueName: \"kubernetes.io/projected/58265cb5-d4cf-45b7-b827-7aa1717c810f-kube-api-access-d9lrb\") pod \"ingress-canary-kns46\" (UID: \"58265cb5-d4cf-45b7-b827-7aa1717c810f\") " pod="openshift-ingress-canary/ingress-canary-kns46" Sep 09 23:18:02.086565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086514 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvx9w\" (UniqueName: \"kubernetes.io/projected/567767c5-25a4-44b6-9f50-446d1ac6d3d6-kube-api-access-nvx9w\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.086565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086531 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b464eccc-dbbb-4c62-9697-a23d8bdb846e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.086565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086558 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/567767c5-25a4-44b6-9f50-446d1ac6d3d6-installation-pull-secrets\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.087164 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086714 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b464eccc-dbbb-4c62-9697-a23d8bdb846e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.087164 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/567767c5-25a4-44b6-9f50-446d1ac6d3d6-trusted-ca\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.087164 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086784 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/567767c5-25a4-44b6-9f50-446d1ac6d3d6-ca-trust-extracted\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.087164 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086828 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/180bec08-a68e-4886-8f1f-126fafcafbc5-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.087164 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086881 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj9c7\" (UniqueName: \"kubernetes.io/projected/b464eccc-dbbb-4c62-9697-a23d8bdb846e-kube-api-access-qj9c7\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.087164 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086915 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/567767c5-25a4-44b6-9f50-446d1ac6d3d6-registry-certificates\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.087164 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.086947 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a33577f-f30a-4592-90e3-2915e9bad561-tmp-dir\") pod \"dns-default-5rjt2\" (UID: \"5a33577f-f30a-4592-90e3-2915e9bad561\") " pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.087874 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.087850 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/567767c5-25a4-44b6-9f50-446d1ac6d3d6-trusted-ca\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.090272 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.090242 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/567767c5-25a4-44b6-9f50-446d1ac6d3d6-installation-pull-secrets\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.090272 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.090247 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/567767c5-25a4-44b6-9f50-446d1ac6d3d6-registry-tls\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.090272 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.090269 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/567767c5-25a4-44b6-9f50-446d1ac6d3d6-image-registry-private-configuration\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.093531 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.093507 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/567767c5-25a4-44b6-9f50-446d1ac6d3d6-bound-sa-token\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.093736 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.093716 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvx9w\" (UniqueName: \"kubernetes.io/projected/567767c5-25a4-44b6-9f50-446d1ac6d3d6-kube-api-access-nvx9w\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.100563 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.100545 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/567767c5-25a4-44b6-9f50-446d1ac6d3d6-registry-certificates\") pod \"image-registry-7b6f97fb9d-wzhqp\" (UID: \"567767c5-25a4-44b6-9f50-446d1ac6d3d6\") " pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.187924 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.187889 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a33577f-f30a-4592-90e3-2915e9bad561-config-volume\") pod \"dns-default-5rjt2\" (UID: \"5a33577f-f30a-4592-90e3-2915e9bad561\") " pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.187924 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.187925 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b464eccc-dbbb-4c62-9697-a23d8bdb846e-data-volume\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.188369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.187943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a33577f-f30a-4592-90e3-2915e9bad561-metrics-tls\") pod \"dns-default-5rjt2\" (UID: \"5a33577f-f30a-4592-90e3-2915e9bad561\") " pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.188369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188078 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/180bec08-a68e-4886-8f1f-126fafcafbc5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.188369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188113 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/180bec08-a68e-4886-8f1f-126fafcafbc5-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.188369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188172 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b464eccc-dbbb-4c62-9697-a23d8bdb846e-crio-socket\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.188369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188211 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twgdq\" (UniqueName: \"kubernetes.io/projected/180bec08-a68e-4886-8f1f-126fafcafbc5-kube-api-access-twgdq\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.188369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shq84\" (UniqueName: \"kubernetes.io/projected/5a33577f-f30a-4592-90e3-2915e9bad561-kube-api-access-shq84\") pod \"dns-default-5rjt2\" (UID: \"5a33577f-f30a-4592-90e3-2915e9bad561\") " pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.188369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188252 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b464eccc-dbbb-4c62-9697-a23d8bdb846e-data-volume\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.188369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188294 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58265cb5-d4cf-45b7-b827-7aa1717c810f-cert\") pod \"ingress-canary-kns46\" (UID: \"58265cb5-d4cf-45b7-b827-7aa1717c810f\") " pod="openshift-ingress-canary/ingress-canary-kns46" Sep 09 23:18:02.188369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188320 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9lrb\" (UniqueName: \"kubernetes.io/projected/58265cb5-d4cf-45b7-b827-7aa1717c810f-kube-api-access-d9lrb\") pod \"ingress-canary-kns46\" (UID: \"58265cb5-d4cf-45b7-b827-7aa1717c810f\") " pod="openshift-ingress-canary/ingress-canary-kns46" Sep 09 23:18:02.188369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b464eccc-dbbb-4c62-9697-a23d8bdb846e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.188843 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b464eccc-dbbb-4c62-9697-a23d8bdb846e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.188843 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188420 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/180bec08-a68e-4886-8f1f-126fafcafbc5-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.188843 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj9c7\" (UniqueName: \"kubernetes.io/projected/b464eccc-dbbb-4c62-9697-a23d8bdb846e-kube-api-access-qj9c7\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.188843 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188471 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b464eccc-dbbb-4c62-9697-a23d8bdb846e-crio-socket\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.188843 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188485 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a33577f-f30a-4592-90e3-2915e9bad561-config-volume\") pod \"dns-default-5rjt2\" (UID: \"5a33577f-f30a-4592-90e3-2915e9bad561\") " pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.188843 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a33577f-f30a-4592-90e3-2915e9bad561-tmp-dir\") pod \"dns-default-5rjt2\" (UID: \"5a33577f-f30a-4592-90e3-2915e9bad561\") " pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.188843 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:18:02.188568 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 09 23:18:02.188843 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:18:02.188627 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/180bec08-a68e-4886-8f1f-126fafcafbc5-prometheus-operator-tls podName:180bec08-a68e-4886-8f1f-126fafcafbc5 nodeName:}" failed. No retries permitted until 2026-09-09 23:18:02.688607516 +0000 UTC m=+44.115030818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/180bec08-a68e-4886-8f1f-126fafcafbc5-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-pjndv" (UID: "180bec08-a68e-4886-8f1f-126fafcafbc5") : secret "prometheus-operator-tls" not found Sep 09 23:18:02.188843 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188830 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/180bec08-a68e-4886-8f1f-126fafcafbc5-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.189329 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.188953 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a33577f-f30a-4592-90e3-2915e9bad561-tmp-dir\") pod \"dns-default-5rjt2\" (UID: \"5a33577f-f30a-4592-90e3-2915e9bad561\") " pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.189329 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.189311 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b464eccc-dbbb-4c62-9697-a23d8bdb846e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.190538 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.190514 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a33577f-f30a-4592-90e3-2915e9bad561-metrics-tls\") pod \"dns-default-5rjt2\" (UID: \"5a33577f-f30a-4592-90e3-2915e9bad561\") " pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.190808 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.190789 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/180bec08-a68e-4886-8f1f-126fafcafbc5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.191235 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.191208 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58265cb5-d4cf-45b7-b827-7aa1717c810f-cert\") pod \"ingress-canary-kns46\" (UID: \"58265cb5-d4cf-45b7-b827-7aa1717c810f\") " pod="openshift-ingress-canary/ingress-canary-kns46" Sep 09 23:18:02.191235 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.191213 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b464eccc-dbbb-4c62-9697-a23d8bdb846e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.197228 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.197205 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shq84\" (UniqueName: \"kubernetes.io/projected/5a33577f-f30a-4592-90e3-2915e9bad561-kube-api-access-shq84\") pod \"dns-default-5rjt2\" (UID: \"5a33577f-f30a-4592-90e3-2915e9bad561\") " pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.197593 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.197575 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twgdq\" (UniqueName: \"kubernetes.io/projected/180bec08-a68e-4886-8f1f-126fafcafbc5-kube-api-access-twgdq\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.197667 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.197609 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9lrb\" (UniqueName: \"kubernetes.io/projected/58265cb5-d4cf-45b7-b827-7aa1717c810f-kube-api-access-d9lrb\") pod \"ingress-canary-kns46\" (UID: \"58265cb5-d4cf-45b7-b827-7aa1717c810f\") " pod="openshift-ingress-canary/ingress-canary-kns46" Sep 09 23:18:02.197938 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.197916 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj9c7\" (UniqueName: \"kubernetes.io/projected/b464eccc-dbbb-4c62-9697-a23d8bdb846e-kube-api-access-qj9c7\") pod \"insights-runtime-extractor-ztk45\" (UID: \"b464eccc-dbbb-4c62-9697-a23d8bdb846e\") " pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.205903 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.205887 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:02.215875 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.215860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:02.237276 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.237245 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kns46" Sep 09 23:18:02.257708 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.257669 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ztk45" Sep 09 23:18:02.502273 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.502117 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5rjt2"] Sep 09 23:18:02.504298 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.504272 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b6f97fb9d-wzhqp"] Sep 09 23:18:02.505183 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:02.505120 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a33577f_f30a_4592_90e3_2915e9bad561.slice/crio-8a6f48dd0e1471b8d2457aa255ec396a4a818b6bd334e26d111aabdf2e925544 WatchSource:0}: Error finding container 8a6f48dd0e1471b8d2457aa255ec396a4a818b6bd334e26d111aabdf2e925544: Status 404 returned error can't find the container with id 8a6f48dd0e1471b8d2457aa255ec396a4a818b6bd334e26d111aabdf2e925544 Sep 09 23:18:02.506953 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:02.506931 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod567767c5_25a4_44b6_9f50_446d1ac6d3d6.slice/crio-4ec85fb3923dbb7dee4a4790580030f7914f23b39765820cf4b29c58bc7d1200 WatchSource:0}: Error finding container 4ec85fb3923dbb7dee4a4790580030f7914f23b39765820cf4b29c58bc7d1200: Status 404 returned error can't find the container with id 4ec85fb3923dbb7dee4a4790580030f7914f23b39765820cf4b29c58bc7d1200 Sep 09 23:18:02.530008 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.529958 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kns46"] Sep 09 23:18:02.533622 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:02.533598 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58265cb5_d4cf_45b7_b827_7aa1717c810f.slice/crio-4edb0cb5ec2a954ba8a140bb50b14d4dd4507e1e46a37c84993942d16b4c29b8 WatchSource:0}: Error finding container 4edb0cb5ec2a954ba8a140bb50b14d4dd4507e1e46a37c84993942d16b4c29b8: Status 404 returned error can't find the container with id 4edb0cb5ec2a954ba8a140bb50b14d4dd4507e1e46a37c84993942d16b4c29b8 Sep 09 23:18:02.543255 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.543231 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ztk45"] Sep 09 23:18:02.552937 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:02.552894 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb464eccc_dbbb_4c62_9697_a23d8bdb846e.slice/crio-be580c2a3edec6ee5bf221d14149992157b6d8e6b489ab6d62e5e56e46c296c1 WatchSource:0}: Error finding container be580c2a3edec6ee5bf221d14149992157b6d8e6b489ab6d62e5e56e46c296c1: Status 404 returned error can't find the container with id be580c2a3edec6ee5bf221d14149992157b6d8e6b489ab6d62e5e56e46c296c1 Sep 09 23:18:02.693628 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.693592 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/180bec08-a68e-4886-8f1f-126fafcafbc5-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.696479 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.696321 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/180bec08-a68e-4886-8f1f-126fafcafbc5-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pjndv\" (UID: \"180bec08-a68e-4886-8f1f-126fafcafbc5\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:02.876312 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:02.876237 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" Sep 09 23:18:03.118456 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.118397 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:18:03.118741 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.118551 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:18:03.119298 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.118397 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:18:03.122364 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.122201 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 23:18:03.122364 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.122215 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 23:18:03.122364 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.122278 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 23:18:03.122576 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.122374 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n28fd\"" Sep 09 23:18:03.122576 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.122513 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-snlxf\"" Sep 09 23:18:03.122576 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.122513 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 23:18:03.153272 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.153246 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pjndv"] Sep 09 23:18:03.159618 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:03.159592 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180bec08_a68e_4886_8f1f_126fafcafbc5.slice/crio-4fe1099734f69b1c2e6dca20c34e44bb53089d04311181ac15af3e110c0d0f3d WatchSource:0}: Error finding container 4fe1099734f69b1c2e6dca20c34e44bb53089d04311181ac15af3e110c0d0f3d: Status 404 returned error can't find the container with id 4fe1099734f69b1c2e6dca20c34e44bb53089d04311181ac15af3e110c0d0f3d Sep 09 23:18:03.342830 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.342788 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" event={"ID":"180bec08-a68e-4886-8f1f-126fafcafbc5","Type":"ContainerStarted","Data":"4fe1099734f69b1c2e6dca20c34e44bb53089d04311181ac15af3e110c0d0f3d"} Sep 09 23:18:03.345420 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.345293 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ztk45" event={"ID":"b464eccc-dbbb-4c62-9697-a23d8bdb846e","Type":"ContainerStarted","Data":"c46a6df6844a3c752cf75f829568072a9425b1d448d27b1f3a95ab0cccc1ba40"} Sep 09 23:18:03.345420 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.345354 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ztk45" event={"ID":"b464eccc-dbbb-4c62-9697-a23d8bdb846e","Type":"ContainerStarted","Data":"be580c2a3edec6ee5bf221d14149992157b6d8e6b489ab6d62e5e56e46c296c1"} Sep 09 23:18:03.348046 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.347995 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kns46" event={"ID":"58265cb5-d4cf-45b7-b827-7aa1717c810f","Type":"ContainerStarted","Data":"4edb0cb5ec2a954ba8a140bb50b14d4dd4507e1e46a37c84993942d16b4c29b8"} Sep 09 23:18:03.350667 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.350087 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" event={"ID":"567767c5-25a4-44b6-9f50-446d1ac6d3d6","Type":"ContainerStarted","Data":"a62a9990e49e964c2b7b3c8b73031fe48bffd0d21aa59e2b8307ff5e70e48c32"} Sep 09 23:18:03.350667 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.350130 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" event={"ID":"567767c5-25a4-44b6-9f50-446d1ac6d3d6","Type":"ContainerStarted","Data":"4ec85fb3923dbb7dee4a4790580030f7914f23b39765820cf4b29c58bc7d1200"} Sep 09 23:18:03.350667 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.350325 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:03.351792 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.351769 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5rjt2" event={"ID":"5a33577f-f30a-4592-90e3-2915e9bad561","Type":"ContainerStarted","Data":"8a6f48dd0e1471b8d2457aa255ec396a4a818b6bd334e26d111aabdf2e925544"} Sep 09 23:18:03.369018 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:03.368951 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" podStartSLOduration=8.368930425 podStartE2EDuration="8.368930425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:17:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:03.367717196 +0000 UTC m=+44.794140504" watchObservedRunningTime="2026-09-09 23:18:03.368930425 +0000 UTC m=+44.795353732" Sep 09 23:18:07.364940 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:07.364736 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" event={"ID":"180bec08-a68e-4886-8f1f-126fafcafbc5","Type":"ContainerStarted","Data":"da54fcf1af42c55caa46e1ddc31dc3f15b43821e796a545d88ebe1571f08a125"} Sep 09 23:18:07.365513 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:07.364984 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" event={"ID":"180bec08-a68e-4886-8f1f-126fafcafbc5","Type":"ContainerStarted","Data":"6028167ee763aa29937ce1afadd34b5ca77d481618fbb483524cd5e50e280db1"} Sep 09 23:18:07.366598 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:07.366570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ztk45" event={"ID":"b464eccc-dbbb-4c62-9697-a23d8bdb846e","Type":"ContainerStarted","Data":"f89bf8f844c282dc8dd02ce807602b157a3d6b26a8533f766e04d6a6ff04b290"} Sep 09 23:18:07.368013 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:07.367985 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kns46" event={"ID":"58265cb5-d4cf-45b7-b827-7aa1717c810f","Type":"ContainerStarted","Data":"23d5c7069b59c199f28dc6fb2b65e6416f55c97ab6bc1e02a9e79d0cc43e8c1c"} Sep 09 23:18:07.369596 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:07.369575 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5rjt2" event={"ID":"5a33577f-f30a-4592-90e3-2915e9bad561","Type":"ContainerStarted","Data":"d93de519e1611e92b35723919574c80620f8d4610b972308ad82ca558203244b"} Sep 09 23:18:07.369664 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:07.369616 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5rjt2" event={"ID":"5a33577f-f30a-4592-90e3-2915e9bad561","Type":"ContainerStarted","Data":"faa7aeae243bbc1da965acf5852e5f5c748fda44fb010487dcd2ecd37c14d21c"} Sep 09 23:18:07.369739 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:07.369723 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:07.380391 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:07.380346 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-pjndv" podStartSLOduration=3.290749018 podStartE2EDuration="6.380335592s" totalImagesPullingTime="3.089586574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:07.379228842 +0000 UTC m=+48.805652150" watchObservedRunningTime="2026-09-09 23:18:07.380335592 +0000 UTC m=+48.806758900" Sep 09 23:18:07.393638 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:07.393600 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5rjt2" podStartSLOduration=2.649531418 podStartE2EDuration="6.393588119s" totalImagesPullingTime="3.744056701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:07.392897802 +0000 UTC m=+48.819321110" watchObservedRunningTime="2026-09-09 23:18:07.393588119 +0000 UTC m=+48.820011426" Sep 09 23:18:07.406783 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:07.406745 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kns46" podStartSLOduration=2.685784974 podStartE2EDuration="6.406732926s" totalImagesPullingTime="3.720947952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:07.406088175 +0000 UTC m=+48.832511478" watchObservedRunningTime="2026-09-09 23:18:07.406732926 +0000 UTC m=+48.833156225" Sep 09 23:18:08.374407 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:08.374376 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ztk45" event={"ID":"b464eccc-dbbb-4c62-9697-a23d8bdb846e","Type":"ContainerStarted","Data":"a818a84abcc035f89361b0f8af7f4e6a3a0e962961287e38036c0d5924e5ccbc"} Sep 09 23:18:09.303982 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.303786 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bz42s"] Sep 09 23:18:09.317226 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.317205 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.319653 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.319633 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bt9rk\"" Sep 09 23:18:09.319853 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.319837 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 23:18:09.319941 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.319854 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 23:18:09.319941 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.319854 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 23:18:09.348711 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.348688 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-accelerators-collector-config\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.348822 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.348717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-wtmp\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.348822 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.348742 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74dccb39-20df-48ef-9138-845b5264dd83-root\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.348935 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.348829 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74dccb39-20df-48ef-9138-845b5264dd83-metrics-client-ca\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.348935 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.348880 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74dccb39-20df-48ef-9138-845b5264dd83-sys\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.349048 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.348936 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-tls\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.349048 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.348977 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljglk\" (UniqueName: \"kubernetes.io/projected/74dccb39-20df-48ef-9138-845b5264dd83-kube-api-access-ljglk\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.349048 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.349038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.349159 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.349055 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-textfile\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.402100 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.402056 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ztk45" podStartSLOduration=3.014658623 podStartE2EDuration="8.402043709s" totalImagesPullingTime="5.387385086s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:18:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:09.401492739 +0000 UTC m=+50.827916050" watchObservedRunningTime="2026-09-09 23:18:09.402043709 +0000 UTC m=+50.828467015" Sep 09 23:18:09.449743 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.449715 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-accelerators-collector-config\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.449866 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.449756 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-wtmp\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.449866 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.449785 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74dccb39-20df-48ef-9138-845b5264dd83-root\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.449866 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.449844 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74dccb39-20df-48ef-9138-845b5264dd83-metrics-client-ca\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.450061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.449883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74dccb39-20df-48ef-9138-845b5264dd83-sys\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.450061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.449919 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74dccb39-20df-48ef-9138-845b5264dd83-root\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.450061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.449923 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-wtmp\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.450061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.449928 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-tls\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.450061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.449985 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74dccb39-20df-48ef-9138-845b5264dd83-sys\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.450061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.450000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljglk\" (UniqueName: \"kubernetes.io/projected/74dccb39-20df-48ef-9138-845b5264dd83-kube-api-access-ljglk\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.450061 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:18:09.450029 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 09 23:18:09.450061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.450054 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.450451 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:18:09.450090 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-tls podName:74dccb39-20df-48ef-9138-845b5264dd83 nodeName:}" failed. No retries permitted until 2026-09-09 23:18:09.950073599 +0000 UTC m=+51.376496901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-tls") pod "node-exporter-bz42s" (UID: "74dccb39-20df-48ef-9138-845b5264dd83") : secret "node-exporter-tls" not found Sep 09 23:18:09.450451 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.450103 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-textfile\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.450451 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.450412 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-accelerators-collector-config\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.450615 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.450486 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74dccb39-20df-48ef-9138-845b5264dd83-metrics-client-ca\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.450615 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.450585 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-textfile\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.454007 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.453986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.460528 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.460508 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljglk\" (UniqueName: \"kubernetes.io/projected/74dccb39-20df-48ef-9138-845b5264dd83-kube-api-access-ljglk\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.953481 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:09.953447 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-tls\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:09.953688 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:18:09.953605 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 09 23:18:09.953688 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:18:09.953679 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-tls podName:74dccb39-20df-48ef-9138-845b5264dd83 nodeName:}" failed. No retries permitted until 2026-09-09 23:18:10.953663795 +0000 UTC m=+52.380087079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-tls") pod "node-exporter-bz42s" (UID: "74dccb39-20df-48ef-9138-845b5264dd83") : secret "node-exporter-tls" not found Sep 09 23:18:10.341405 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.341369 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 23:18:10.355810 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.355785 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.359507 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.358944 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 09 23:18:10.359507 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.358958 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 09 23:18:10.359507 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.359268 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 09 23:18:10.359507 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.359349 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 09 23:18:10.361442 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.361367 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 09 23:18:10.361442 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.361390 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-l8fml\"" Sep 09 23:18:10.361442 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.361409 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 09 23:18:10.361674 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.361496 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 09 23:18:10.361674 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.361621 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 09 23:18:10.361958 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.361796 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 09 23:18:10.361958 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.361834 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 23:18:10.435569 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.435530 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-596f448855-9h778"] Sep 09 23:18:10.457313 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457284 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb748af0-56d4-4d5b-8193-735bdbdb85bc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457442 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457329 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-web-config\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457442 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457442 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457442 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457423 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457628 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457628 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457482 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb748af0-56d4-4d5b-8193-735bdbdb85bc-config-out\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457628 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457628 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457519 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457628 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457548 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457628 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457591 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdf67\" (UniqueName: \"kubernetes.io/projected/fb748af0-56d4-4d5b-8193-735bdbdb85bc-kube-api-access-bdf67\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457831 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457628 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457831 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.457831 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457672 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-596f448855-9h778"] Sep 09 23:18:10.457831 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.457764 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.460444 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.460424 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 09 23:18:10.460573 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.460492 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 09 23:18:10.460573 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.460454 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 09 23:18:10.460715 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.460681 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-g7t5s\"" Sep 09 23:18:10.460775 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.460730 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 09 23:18:10.460775 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.460769 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 09 23:18:10.461005 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.460987 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 09 23:18:10.461097 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.460995 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 09 23:18:10.465441 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.465421 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 09 23:18:10.558601 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.558569 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-trusted-ca-bundle\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.558601 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.558600 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-serving-cert\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.558797 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.558623 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-oauth-serving-cert\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.558797 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.558661 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-service-ca\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.558797 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.558699 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-oauth-config\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.558797 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.558718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdf67\" (UniqueName: \"kubernetes.io/projected/fb748af0-56d4-4d5b-8193-735bdbdb85bc-kube-api-access-bdf67\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.558797 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.558736 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.558797 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.558758 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn787\" (UniqueName: \"kubernetes.io/projected/68e3b494-ecda-49ca-a06d-5d941dea78c7-kube-api-access-nn787\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.558797 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.558786 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.559242 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.558805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb748af0-56d4-4d5b-8193-735bdbdb85bc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.559242 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:18:10.558913 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-trusted-ca-bundle podName:fb748af0-56d4-4d5b-8193-735bdbdb85bc nodeName:}" failed. No retries permitted until 2026-09-09 23:18:11.058897708 +0000 UTC m=+52.485320999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc") : configmap references non-existent config key: ca-bundle.crt Sep 09 23:18:10.559242 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.558953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-web-config\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.559242 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.559011 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.559242 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.559049 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-config\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.559242 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.559081 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.559242 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.559114 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.559242 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.559141 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.559242 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.559206 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb748af0-56d4-4d5b-8193-735bdbdb85bc-config-out\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.559242 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.559234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.559693 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.559261 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.559693 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.559296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.559693 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.559480 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.560310 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.560286 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.562088 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.562045 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.562221 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.562203 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb748af0-56d4-4d5b-8193-735bdbdb85bc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.562768 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.562464 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb748af0-56d4-4d5b-8193-735bdbdb85bc-config-out\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.562768 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.562603 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-web-config\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.562768 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.562730 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.562768 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.562752 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.562929 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.562817 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.563802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.563781 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.563887 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.563854 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.566488 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.566465 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdf67\" (UniqueName: \"kubernetes.io/projected/fb748af0-56d4-4d5b-8193-735bdbdb85bc-kube-api-access-bdf67\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:10.660615 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.660546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-trusted-ca-bundle\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.660615 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.660577 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-serving-cert\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.660615 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.660594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-oauth-serving-cert\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.660615 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.660616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-service-ca\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.660931 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.660633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-oauth-config\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.660931 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.660651 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn787\" (UniqueName: \"kubernetes.io/projected/68e3b494-ecda-49ca-a06d-5d941dea78c7-kube-api-access-nn787\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.660931 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.660721 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-config\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.661389 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.661366 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-oauth-serving-cert\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.661480 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.661428 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-trusted-ca-bundle\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.661480 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.661454 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-service-ca\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.661480 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.661471 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-config\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.663579 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.663560 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-serving-cert\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.663656 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.663577 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-oauth-config\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.668563 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.668541 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn787\" (UniqueName: \"kubernetes.io/projected/68e3b494-ecda-49ca-a06d-5d941dea78c7-kube-api-access-nn787\") pod \"console-596f448855-9h778\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.767727 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.767699 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:10.963480 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.963375 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-tls\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:10.966048 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:10.966026 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74dccb39-20df-48ef-9138-845b5264dd83-node-exporter-tls\") pod \"node-exporter-bz42s\" (UID: \"74dccb39-20df-48ef-9138-845b5264dd83\") " pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:11.035602 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:11.035563 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-596f448855-9h778"] Sep 09 23:18:11.039222 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:11.039195 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e3b494_ecda_49ca_a06d_5d941dea78c7.slice/crio-b37f39ad8998d8f535d0ed3a6518ede661650472efa142ac9318250c30ae566f WatchSource:0}: Error finding container b37f39ad8998d8f535d0ed3a6518ede661650472efa142ac9318250c30ae566f: Status 404 returned error can't find the container with id b37f39ad8998d8f535d0ed3a6518ede661650472efa142ac9318250c30ae566f Sep 09 23:18:11.064578 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:11.064552 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:11.065678 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:11.065652 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:11.125886 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:11.125859 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bz42s" Sep 09 23:18:11.135329 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:11.135306 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74dccb39_20df_48ef_9138_845b5264dd83.slice/crio-28bc874ec65f93818882907c60a920da0a2e778af6978839418ecad1e3691a26 WatchSource:0}: Error finding container 28bc874ec65f93818882907c60a920da0a2e778af6978839418ecad1e3691a26: Status 404 returned error can't find the container with id 28bc874ec65f93818882907c60a920da0a2e778af6978839418ecad1e3691a26 Sep 09 23:18:11.267678 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:11.267604 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:18:11.387349 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:11.386982 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bz42s" event={"ID":"74dccb39-20df-48ef-9138-845b5264dd83","Type":"ContainerStarted","Data":"28bc874ec65f93818882907c60a920da0a2e778af6978839418ecad1e3691a26"} Sep 09 23:18:11.389083 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:11.389061 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-596f448855-9h778" event={"ID":"68e3b494-ecda-49ca-a06d-5d941dea78c7","Type":"ContainerStarted","Data":"b37f39ad8998d8f535d0ed3a6518ede661650472efa142ac9318250c30ae566f"} Sep 09 23:18:11.546549 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:11.546521 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 23:18:11.548581 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:11.548553 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb748af0_56d4_4d5b_8193_735bdbdb85bc.slice/crio-a339c78d2bf6dec667fcdf34b2499ce0ceb5e1f7d223ea4c7b26658be5153922 WatchSource:0}: Error finding container a339c78d2bf6dec667fcdf34b2499ce0ceb5e1f7d223ea4c7b26658be5153922: Status 404 returned error can't find the container with id a339c78d2bf6dec667fcdf34b2499ce0ceb5e1f7d223ea4c7b26658be5153922 Sep 09 23:18:12.401711 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:12.401663 2580 generic.go:356] "Generic (PLEG): container finished" podID="74dccb39-20df-48ef-9138-845b5264dd83" containerID="4170e1dc94970365de49c1bd7a5cc9dac1ecdf15140d029c2a93b5b429fbbca0" exitCode=0 Sep 09 23:18:12.401876 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:12.401733 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bz42s" event={"ID":"74dccb39-20df-48ef-9138-845b5264dd83","Type":"ContainerDied","Data":"4170e1dc94970365de49c1bd7a5cc9dac1ecdf15140d029c2a93b5b429fbbca0"} Sep 09 23:18:12.403210 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:12.403186 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerStarted","Data":"a339c78d2bf6dec667fcdf34b2499ce0ceb5e1f7d223ea4c7b26658be5153922"} Sep 09 23:18:13.411037 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.411000 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bz42s" event={"ID":"74dccb39-20df-48ef-9138-845b5264dd83","Type":"ContainerStarted","Data":"d34b78b41788d1824d63c4d776d850ca26cdb36f0f366657b6054ea4bda145e1"} Sep 09 23:18:13.411678 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.411075 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bz42s" event={"ID":"74dccb39-20df-48ef-9138-845b5264dd83","Type":"ContainerStarted","Data":"4506a1e6669dbdb5f519c4ac389c3341777f0e75c35afe5f12a65274c0723230"} Sep 09 23:18:13.431039 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.430997 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bz42s" podStartSLOduration=3.300033675 podStartE2EDuration="4.430981916s" totalImagesPullingTime="1.058578449s" totalInitContainerRuntime="72.369792ms" isStatefulPod=true podCreationTimestamp="2026-09-09 23:18:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:13.429327451 +0000 UTC m=+54.855750762" watchObservedRunningTime="2026-09-09 23:18:13.430981916 +0000 UTC m=+54.857405214" Sep 09 23:18:13.582319 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.582287 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-56fd4848c7-5zb6j"] Sep 09 23:18:13.586142 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.586111 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.588525 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.588498 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-sw8bf\"" Sep 09 23:18:13.588652 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.588610 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8588vsidlc92h\"" Sep 09 23:18:13.588721 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.588672 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 09 23:18:13.588772 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.588749 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 09 23:18:13.588850 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.588834 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 09 23:18:13.589034 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.589013 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 09 23:18:13.595615 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.595563 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56fd4848c7-5zb6j"] Sep 09 23:18:13.690077 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.689994 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/522c3955-f498-45d8-85c9-852d52ff4efc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.690077 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.690045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/522c3955-f498-45d8-85c9-852d52ff4efc-metrics-server-audit-profiles\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.690292 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.690110 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/522c3955-f498-45d8-85c9-852d52ff4efc-secret-metrics-server-client-certs\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.690292 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.690146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/522c3955-f498-45d8-85c9-852d52ff4efc-client-ca-bundle\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.690292 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.690181 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bndlr\" (UniqueName: \"kubernetes.io/projected/522c3955-f498-45d8-85c9-852d52ff4efc-kube-api-access-bndlr\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.690292 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.690275 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/522c3955-f498-45d8-85c9-852d52ff4efc-secret-metrics-server-tls\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.690436 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.690337 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/522c3955-f498-45d8-85c9-852d52ff4efc-audit-log\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.791228 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.791192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/522c3955-f498-45d8-85c9-852d52ff4efc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.791408 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.791236 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/522c3955-f498-45d8-85c9-852d52ff4efc-metrics-server-audit-profiles\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.791408 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.791288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/522c3955-f498-45d8-85c9-852d52ff4efc-secret-metrics-server-client-certs\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.791408 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.791320 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/522c3955-f498-45d8-85c9-852d52ff4efc-client-ca-bundle\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.791564 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.791443 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bndlr\" (UniqueName: \"kubernetes.io/projected/522c3955-f498-45d8-85c9-852d52ff4efc-kube-api-access-bndlr\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.791564 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.791508 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/522c3955-f498-45d8-85c9-852d52ff4efc-secret-metrics-server-tls\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.791564 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.791537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/522c3955-f498-45d8-85c9-852d52ff4efc-audit-log\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.791989 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.791903 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/522c3955-f498-45d8-85c9-852d52ff4efc-audit-log\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.792119 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.791985 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/522c3955-f498-45d8-85c9-852d52ff4efc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.792333 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.792311 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/522c3955-f498-45d8-85c9-852d52ff4efc-metrics-server-audit-profiles\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.795285 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.795268 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/522c3955-f498-45d8-85c9-852d52ff4efc-secret-metrics-server-tls\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.795364 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.795349 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/522c3955-f498-45d8-85c9-852d52ff4efc-secret-metrics-server-client-certs\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.795499 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.795473 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/522c3955-f498-45d8-85c9-852d52ff4efc-client-ca-bundle\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.799372 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.799351 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bndlr\" (UniqueName: \"kubernetes.io/projected/522c3955-f498-45d8-85c9-852d52ff4efc-kube-api-access-bndlr\") pod \"metrics-server-56fd4848c7-5zb6j\" (UID: \"522c3955-f498-45d8-85c9-852d52ff4efc\") " pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:13.897343 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:13.897303 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:14.296553 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:14.296525 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56fd4848c7-5zb6j"] Sep 09 23:18:14.299180 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:14.299159 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod522c3955_f498_45d8_85c9_852d52ff4efc.slice/crio-71cddb1ad69643a73027c3b7d160dddc0ff0f18506e812bbab813ca56ffe5993 WatchSource:0}: Error finding container 71cddb1ad69643a73027c3b7d160dddc0ff0f18506e812bbab813ca56ffe5993: Status 404 returned error can't find the container with id 71cddb1ad69643a73027c3b7d160dddc0ff0f18506e812bbab813ca56ffe5993 Sep 09 23:18:14.415761 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:14.415727 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-596f448855-9h778" event={"ID":"68e3b494-ecda-49ca-a06d-5d941dea78c7","Type":"ContainerStarted","Data":"1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519"} Sep 09 23:18:14.417473 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:14.417451 2580 generic.go:356] "Generic (PLEG): container finished" podID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerID="40f807149701b243df30aaaf374a51ec98bf4ebb318d8665275a389542acaba5" exitCode=0 Sep 09 23:18:14.417583 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:14.417515 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerDied","Data":"40f807149701b243df30aaaf374a51ec98bf4ebb318d8665275a389542acaba5"} Sep 09 23:18:14.418732 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:14.418706 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" event={"ID":"522c3955-f498-45d8-85c9-852d52ff4efc","Type":"ContainerStarted","Data":"71cddb1ad69643a73027c3b7d160dddc0ff0f18506e812bbab813ca56ffe5993"} Sep 09 23:18:14.431691 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:14.431655 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-596f448855-9h778" podStartSLOduration=1.4378624549999999 podStartE2EDuration="4.431641574s" totalImagesPullingTime="2.993779119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:18:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:14.430633919 +0000 UTC m=+55.857057240" watchObservedRunningTime="2026-09-09 23:18:14.431641574 +0000 UTC m=+55.858064882" Sep 09 23:18:15.667193 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.667157 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-596f448855-9h778"] Sep 09 23:18:15.702395 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.701024 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-759fc57545-68zh8"] Sep 09 23:18:15.704715 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.704034 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.726227 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.726156 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-759fc57545-68zh8"] Sep 09 23:18:15.812790 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.812490 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-service-ca\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.812790 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.812536 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6e79a72-f526-42a7-9507-38c57161b457-console-serving-cert\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.812790 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.812565 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-oauth-serving-cert\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.812790 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.812609 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sghc\" (UniqueName: \"kubernetes.io/projected/a6e79a72-f526-42a7-9507-38c57161b457-kube-api-access-5sghc\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.812790 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.812664 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6e79a72-f526-42a7-9507-38c57161b457-console-oauth-config\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.812790 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.812703 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-trusted-ca-bundle\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.812790 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.812746 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-console-config\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.914080 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.914041 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-service-ca\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.914295 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.914089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6e79a72-f526-42a7-9507-38c57161b457-console-serving-cert\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.914295 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.914120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-oauth-serving-cert\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.914295 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.914139 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sghc\" (UniqueName: \"kubernetes.io/projected/a6e79a72-f526-42a7-9507-38c57161b457-kube-api-access-5sghc\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.914295 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.914163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6e79a72-f526-42a7-9507-38c57161b457-console-oauth-config\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.914295 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.914200 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-trusted-ca-bundle\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.914295 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.914240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-console-config\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.914941 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.914906 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-oauth-serving-cert\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.915066 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.914915 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-service-ca\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.915066 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.914953 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-console-config\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.915431 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.915407 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-trusted-ca-bundle\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.917258 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.917186 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6e79a72-f526-42a7-9507-38c57161b457-console-serving-cert\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.917258 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.917191 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6e79a72-f526-42a7-9507-38c57161b457-console-oauth-config\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:15.923156 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:15.923136 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sghc\" (UniqueName: \"kubernetes.io/projected/a6e79a72-f526-42a7-9507-38c57161b457-kube-api-access-5sghc\") pod \"console-759fc57545-68zh8\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:16.017915 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:16.017873 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:16.926316 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:16.926246 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-759fc57545-68zh8"] Sep 09 23:18:16.928961 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:16.928898 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6e79a72_f526_42a7_9507_38c57161b457.slice/crio-873419cb9e01fbb6eef09d05a2414ceea7424911aeed91b024cadc87d99cbe68 WatchSource:0}: Error finding container 873419cb9e01fbb6eef09d05a2414ceea7424911aeed91b024cadc87d99cbe68: Status 404 returned error can't find the container with id 873419cb9e01fbb6eef09d05a2414ceea7424911aeed91b024cadc87d99cbe68 Sep 09 23:18:17.302617 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:17.302589 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-66lv5" Sep 09 23:18:17.376801 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:17.376775 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5rjt2" Sep 09 23:18:17.434438 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:17.434405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerStarted","Data":"2e00b81a7c721432f17f72990b81d1e6beae0ab077087161b4d4fceb3ac2c589"} Sep 09 23:18:17.434601 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:17.434484 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerStarted","Data":"0f077b3f7d7e0a69b42ef94e6b1fbc020fbb82abc323121313a4685bfd5559e3"} Sep 09 23:18:17.434601 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:17.434530 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerStarted","Data":"ce87d4a62e4652e3ff69222eda32d4bd827fea650ec7fd1d0ee72f3920f48d8e"} Sep 09 23:18:17.434601 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:17.434568 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerStarted","Data":"05625f67b6dcd0f25f9066162c4c12df8cb8376ef810607b68bb6ad868db5baa"} Sep 09 23:18:17.434756 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:17.434606 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerStarted","Data":"a29c2ba94b1c5c9ec6c49c2623b5269b67c7bdce630ff2b16490cfd25923434b"} Sep 09 23:18:17.436203 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:17.436174 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" event={"ID":"522c3955-f498-45d8-85c9-852d52ff4efc","Type":"ContainerStarted","Data":"191bdf1cacdb064760e07ba0bad69c50606acc77452ab5a2506bcb5ab00f7e6b"} Sep 09 23:18:17.438832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:17.438805 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-759fc57545-68zh8" event={"ID":"a6e79a72-f526-42a7-9507-38c57161b457","Type":"ContainerStarted","Data":"84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6"} Sep 09 23:18:17.438911 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:17.438839 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-759fc57545-68zh8" event={"ID":"a6e79a72-f526-42a7-9507-38c57161b457","Type":"ContainerStarted","Data":"873419cb9e01fbb6eef09d05a2414ceea7424911aeed91b024cadc87d99cbe68"} Sep 09 23:18:17.452743 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:17.452703 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" podStartSLOduration=2.090579348 podStartE2EDuration="4.452689379s" totalImagesPullingTime="2.362110031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:17.45111037 +0000 UTC m=+58.877533678" watchObservedRunningTime="2026-09-09 23:18:17.452689379 +0000 UTC m=+58.879112757" Sep 09 23:18:17.468242 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:17.468200 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-759fc57545-68zh8" podStartSLOduration=2.46817228 podStartE2EDuration="2.46817228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:18:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:17.466704466 +0000 UTC m=+58.893127775" watchObservedRunningTime="2026-09-09 23:18:17.46817228 +0000 UTC m=+58.894595591" Sep 09 23:18:18.445827 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:18.445788 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerStarted","Data":"0d4f8fd807d94ef56448d018cc46400da7aaabc599e6f94ebed2d22798effa59"} Sep 09 23:18:18.474427 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:18.474376 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.983551592 podStartE2EDuration="8.474360918s" totalImagesPullingTime="5.469090176s" totalInitContainerRuntime="21.71915ms" isStatefulPod=false podCreationTimestamp="2026-09-09 23:18:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:18.469846241 +0000 UTC m=+59.896269548" watchObservedRunningTime="2026-09-09 23:18:18.474360918 +0000 UTC m=+59.900784258" Sep 09 23:18:20.768729 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:20.768688 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:24.359813 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:24.359771 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b6f97fb9d-wzhqp" Sep 09 23:18:24.797018 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:24.796983 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:18:24.797192 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:24.797140 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:18:24.799624 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:24.799606 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 23:18:24.799696 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:24.799606 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 23:18:24.810103 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:24.810079 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b326930-e182-4769-b41a-f79a91a2a5db-original-pull-secret\") pod \"global-pull-secret-syncer-xxk7v\" (UID: \"6b326930-e182-4769-b41a-f79a91a2a5db\") " pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:18:24.810192 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:24.810161 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63dd3bbf-4802-4b4a-9b06-37d9cf22ee94-metrics-certs\") pod \"network-metrics-daemon-vdx58\" (UID: \"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94\") " pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:18:24.998534 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:24.998496 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5hf2\" (UniqueName: \"kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2\") pod \"network-check-target-mr82q\" (UID: \"fbf8f0ca-22cf-4c56-89df-8a94b1ac7795\") " pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:18:25.000824 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.000802 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 23:18:25.011795 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.011776 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 23:18:25.022347 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.022326 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5hf2\" (UniqueName: \"kubernetes.io/projected/fbf8f0ca-22cf-4c56-89df-8a94b1ac7795-kube-api-access-l5hf2\") pod \"network-check-target-mr82q\" (UID: \"fbf8f0ca-22cf-4c56-89df-8a94b1ac7795\") " pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:18:25.038526 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.038501 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-snlxf\"" Sep 09 23:18:25.045445 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.045425 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n28fd\"" Sep 09 23:18:25.046479 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.046457 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:18:25.050162 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.050146 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxk7v" Sep 09 23:18:25.053677 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.053656 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vdx58" Sep 09 23:18:25.325684 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.325658 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mr82q"] Sep 09 23:18:25.328371 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:25.328342 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf8f0ca_22cf_4c56_89df_8a94b1ac7795.slice/crio-287905e8e6d4385a9a22d87167e07117e2e5cb71928f109d2dc0c874b3b186a3 WatchSource:0}: Error finding container 287905e8e6d4385a9a22d87167e07117e2e5cb71928f109d2dc0c874b3b186a3: Status 404 returned error can't find the container with id 287905e8e6d4385a9a22d87167e07117e2e5cb71928f109d2dc0c874b3b186a3 Sep 09 23:18:25.328929 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.328907 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vdx58"] Sep 09 23:18:25.331558 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.331534 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xxk7v"] Sep 09 23:18:25.331840 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:25.331815 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63dd3bbf_4802_4b4a_9b06_37d9cf22ee94.slice/crio-e6e0762f500c1982955da1e2d0fdd3522d0981c44380a4e4268f8595daa90075 WatchSource:0}: Error finding container e6e0762f500c1982955da1e2d0fdd3522d0981c44380a4e4268f8595daa90075: Status 404 returned error can't find the container with id e6e0762f500c1982955da1e2d0fdd3522d0981c44380a4e4268f8595daa90075 Sep 09 23:18:25.335704 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:18:25.335681 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b326930_e182_4769_b41a_f79a91a2a5db.slice/crio-2fac236c8af5717973ea8610d895c362652a9c7cb7be64a21902230124bc4852 WatchSource:0}: Error finding container 2fac236c8af5717973ea8610d895c362652a9c7cb7be64a21902230124bc4852: Status 404 returned error can't find the container with id 2fac236c8af5717973ea8610d895c362652a9c7cb7be64a21902230124bc4852 Sep 09 23:18:25.471333 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.471293 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mr82q" event={"ID":"fbf8f0ca-22cf-4c56-89df-8a94b1ac7795","Type":"ContainerStarted","Data":"287905e8e6d4385a9a22d87167e07117e2e5cb71928f109d2dc0c874b3b186a3"} Sep 09 23:18:25.472468 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.472445 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vdx58" event={"ID":"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94","Type":"ContainerStarted","Data":"e6e0762f500c1982955da1e2d0fdd3522d0981c44380a4e4268f8595daa90075"} Sep 09 23:18:25.473571 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:25.473553 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xxk7v" event={"ID":"6b326930-e182-4769-b41a-f79a91a2a5db","Type":"ContainerStarted","Data":"2fac236c8af5717973ea8610d895c362652a9c7cb7be64a21902230124bc4852"} Sep 09 23:18:26.019018 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:26.018895 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:26.019018 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:26.018984 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:26.025801 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:26.025738 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:26.484356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:26.484325 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:18:27.485516 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:27.485478 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vdx58" event={"ID":"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94","Type":"ContainerStarted","Data":"b1bda7f4f87f2f8a1241f400ee7dd37d1b5c8225c26d8978d70de1e11db3971b"} Sep 09 23:18:27.485956 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:27.485541 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vdx58" event={"ID":"63dd3bbf-4802-4b4a-9b06-37d9cf22ee94","Type":"ContainerStarted","Data":"65d009c97fb5b4a9f5b3efc3d5051001f42f842f542c342ebafd36115fce5aeb"} Sep 09 23:18:27.500849 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:27.500796 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vdx58" podStartSLOduration=67.383568783 podStartE2EDuration="1m8.500778233s" totalImagesPullingTime="1.11720945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:27.499670074 +0000 UTC m=+68.926093384" watchObservedRunningTime="2026-09-09 23:18:27.500778233 +0000 UTC m=+68.927201537" Sep 09 23:18:30.497596 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:30.497558 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mr82q" event={"ID":"fbf8f0ca-22cf-4c56-89df-8a94b1ac7795","Type":"ContainerStarted","Data":"95100b514073672ba17da92e9bce41c416d937043014f47cb1d0c777ab8a0602"} Sep 09 23:18:30.498029 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:30.497671 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:18:30.499067 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:30.499045 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xxk7v" event={"ID":"6b326930-e182-4769-b41a-f79a91a2a5db","Type":"ContainerStarted","Data":"9e63521cd48a6491959634475a9236cf8e6bd34de0e7ab0baeac89c885a8295d"} Sep 09 23:18:30.515090 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:30.515049 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mr82q" podStartSLOduration=66.901948533 podStartE2EDuration="1m11.515035197s" totalImagesPullingTime="4.613086664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:30.513668442 +0000 UTC m=+71.940091750" watchObservedRunningTime="2026-09-09 23:18:30.515035197 +0000 UTC m=+71.941458506" Sep 09 23:18:30.528714 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:30.528675 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xxk7v" podStartSLOduration=66.918621981 podStartE2EDuration="1m11.528661988s" totalImagesPullingTime="4.610040007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:30.527942281 +0000 UTC m=+71.954365589" watchObservedRunningTime="2026-09-09 23:18:30.528661988 +0000 UTC m=+71.955085298" Sep 09 23:18:33.897783 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:33.897747 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:33.898192 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:33.897807 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:41.446601 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.446476 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-596f448855-9h778" podUID="68e3b494-ecda-49ca-a06d-5d941dea78c7" containerName="console" containerID="cri-o://1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519" gracePeriod=15 Sep 09 23:18:41.681809 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.681785 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:41.833029 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.833001 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-config\") pod \"68e3b494-ecda-49ca-a06d-5d941dea78c7\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " Sep 09 23:18:41.833185 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.833046 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-trusted-ca-bundle\") pod \"68e3b494-ecda-49ca-a06d-5d941dea78c7\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " Sep 09 23:18:41.833185 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.833065 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-service-ca\") pod \"68e3b494-ecda-49ca-a06d-5d941dea78c7\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " Sep 09 23:18:41.833185 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.833094 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-oauth-config\") pod \"68e3b494-ecda-49ca-a06d-5d941dea78c7\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " Sep 09 23:18:41.833185 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.833130 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-oauth-serving-cert\") pod \"68e3b494-ecda-49ca-a06d-5d941dea78c7\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " Sep 09 23:18:41.833185 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.833154 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-serving-cert\") pod \"68e3b494-ecda-49ca-a06d-5d941dea78c7\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " Sep 09 23:18:41.833185 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.833172 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nn787\" (UniqueName: \"kubernetes.io/projected/68e3b494-ecda-49ca-a06d-5d941dea78c7-kube-api-access-nn787\") pod \"68e3b494-ecda-49ca-a06d-5d941dea78c7\" (UID: \"68e3b494-ecda-49ca-a06d-5d941dea78c7\") " Sep 09 23:18:41.833540 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.833504 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "68e3b494-ecda-49ca-a06d-5d941dea78c7" (UID: "68e3b494-ecda-49ca-a06d-5d941dea78c7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:18:41.833680 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.833571 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-service-ca" (OuterVolumeSpecName: "service-ca") pod "68e3b494-ecda-49ca-a06d-5d941dea78c7" (UID: "68e3b494-ecda-49ca-a06d-5d941dea78c7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:18:41.833743 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.833729 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-config" (OuterVolumeSpecName: "console-config") pod "68e3b494-ecda-49ca-a06d-5d941dea78c7" (UID: "68e3b494-ecda-49ca-a06d-5d941dea78c7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:18:41.833799 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.833737 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "68e3b494-ecda-49ca-a06d-5d941dea78c7" (UID: "68e3b494-ecda-49ca-a06d-5d941dea78c7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:18:41.835559 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.835535 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68e3b494-ecda-49ca-a06d-5d941dea78c7-kube-api-access-nn787" (OuterVolumeSpecName: "kube-api-access-nn787") pod "68e3b494-ecda-49ca-a06d-5d941dea78c7" (UID: "68e3b494-ecda-49ca-a06d-5d941dea78c7"). InnerVolumeSpecName "kube-api-access-nn787". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:18:41.835648 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.835542 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "68e3b494-ecda-49ca-a06d-5d941dea78c7" (UID: "68e3b494-ecda-49ca-a06d-5d941dea78c7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:18:41.835648 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.835612 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "68e3b494-ecda-49ca-a06d-5d941dea78c7" (UID: "68e3b494-ecda-49ca-a06d-5d941dea78c7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:18:41.934207 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.934164 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-oauth-config\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:18:41.934207 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.934203 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-oauth-serving-cert\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:18:41.934207 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.934216 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-serving-cert\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:18:41.934400 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.934229 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn787\" (UniqueName: \"kubernetes.io/projected/68e3b494-ecda-49ca-a06d-5d941dea78c7-kube-api-access-nn787\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:18:41.934400 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.934250 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-console-config\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:18:41.934400 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.934262 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-trusted-ca-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:18:41.934400 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:41.934273 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68e3b494-ecda-49ca-a06d-5d941dea78c7-service-ca\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:18:42.545754 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:42.545717 2580 generic.go:356] "Generic (PLEG): container finished" podID="68e3b494-ecda-49ca-a06d-5d941dea78c7" containerID="1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519" exitCode=0 Sep 09 23:18:42.546221 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:42.545760 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-596f448855-9h778" Sep 09 23:18:42.546221 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:42.545772 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-596f448855-9h778" event={"ID":"68e3b494-ecda-49ca-a06d-5d941dea78c7","Type":"ContainerDied","Data":"1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519"} Sep 09 23:18:42.546221 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:42.545823 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-596f448855-9h778" event={"ID":"68e3b494-ecda-49ca-a06d-5d941dea78c7","Type":"ContainerDied","Data":"b37f39ad8998d8f535d0ed3a6518ede661650472efa142ac9318250c30ae566f"} Sep 09 23:18:42.546221 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:42.545846 2580 scope.go:117] "RemoveContainer" containerID="1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519" Sep 09 23:18:42.555027 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:42.555007 2580 scope.go:117] "RemoveContainer" containerID="1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519" Sep 09 23:18:42.555307 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:18:42.555289 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519\": container with ID starting with 1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519 not found: ID does not exist" containerID="1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519" Sep 09 23:18:42.555372 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:42.555318 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519"} err="failed to get container status \"1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519\": rpc error: code = NotFound desc = could not find container \"1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519\": container with ID starting with 1295fc8c83dd6f074a2fe236e816886e1575293b3666a449a0b5d410343bb519 not found: ID does not exist" Sep 09 23:18:42.566045 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:42.566023 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-596f448855-9h778"] Sep 09 23:18:42.569261 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:42.569241 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-596f448855-9h778"] Sep 09 23:18:43.122742 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:43.122708 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68e3b494-ecda-49ca-a06d-5d941dea78c7" path="/var/lib/kubelet/pods/68e3b494-ecda-49ca-a06d-5d941dea78c7/volumes" Sep 09 23:18:53.903694 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:53.903666 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:18:53.907547 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:18:53.907525 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-56fd4848c7-5zb6j" Sep 09 23:19:01.505876 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:01.505843 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mr82q" Sep 09 23:19:29.510483 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.510444 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 23:19:29.510997 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.510774 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="alertmanager" containerID="cri-o://a29c2ba94b1c5c9ec6c49c2623b5269b67c7bdce630ff2b16490cfd25923434b" gracePeriod=120 Sep 09 23:19:29.510997 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.510848 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="kube-rbac-proxy-metric" containerID="cri-o://2e00b81a7c721432f17f72990b81d1e6beae0ab077087161b4d4fceb3ac2c589" gracePeriod=120 Sep 09 23:19:29.510997 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.510882 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="kube-rbac-proxy-web" containerID="cri-o://ce87d4a62e4652e3ff69222eda32d4bd827fea650ec7fd1d0ee72f3920f48d8e" gracePeriod=120 Sep 09 23:19:29.510997 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.510892 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="prom-label-proxy" containerID="cri-o://0d4f8fd807d94ef56448d018cc46400da7aaabc599e6f94ebed2d22798effa59" gracePeriod=120 Sep 09 23:19:29.510997 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.510949 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="kube-rbac-proxy" containerID="cri-o://0f077b3f7d7e0a69b42ef94e6b1fbc020fbb82abc323121313a4685bfd5559e3" gracePeriod=120 Sep 09 23:19:29.511272 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.510906 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="config-reloader" containerID="cri-o://05625f67b6dcd0f25f9066162c4c12df8cb8376ef810607b68bb6ad868db5baa" gracePeriod=120 Sep 09 23:19:29.721028 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.720998 2580 generic.go:356] "Generic (PLEG): container finished" podID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerID="0d4f8fd807d94ef56448d018cc46400da7aaabc599e6f94ebed2d22798effa59" exitCode=0 Sep 09 23:19:29.721028 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.721023 2580 generic.go:356] "Generic (PLEG): container finished" podID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerID="0f077b3f7d7e0a69b42ef94e6b1fbc020fbb82abc323121313a4685bfd5559e3" exitCode=0 Sep 09 23:19:29.721028 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.721030 2580 generic.go:356] "Generic (PLEG): container finished" podID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerID="05625f67b6dcd0f25f9066162c4c12df8cb8376ef810607b68bb6ad868db5baa" exitCode=0 Sep 09 23:19:29.721215 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.721036 2580 generic.go:356] "Generic (PLEG): container finished" podID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerID="a29c2ba94b1c5c9ec6c49c2623b5269b67c7bdce630ff2b16490cfd25923434b" exitCode=0 Sep 09 23:19:29.721215 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.721067 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerDied","Data":"0d4f8fd807d94ef56448d018cc46400da7aaabc599e6f94ebed2d22798effa59"} Sep 09 23:19:29.721215 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.721129 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerDied","Data":"0f077b3f7d7e0a69b42ef94e6b1fbc020fbb82abc323121313a4685bfd5559e3"} Sep 09 23:19:29.721215 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.721163 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerDied","Data":"05625f67b6dcd0f25f9066162c4c12df8cb8376ef810607b68bb6ad868db5baa"} Sep 09 23:19:29.721215 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:29.721188 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerDied","Data":"a29c2ba94b1c5c9ec6c49c2623b5269b67c7bdce630ff2b16490cfd25923434b"} Sep 09 23:19:30.728385 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.728357 2580 generic.go:356] "Generic (PLEG): container finished" podID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerID="2e00b81a7c721432f17f72990b81d1e6beae0ab077087161b4d4fceb3ac2c589" exitCode=0 Sep 09 23:19:30.728385 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.728380 2580 generic.go:356] "Generic (PLEG): container finished" podID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerID="ce87d4a62e4652e3ff69222eda32d4bd827fea650ec7fd1d0ee72f3920f48d8e" exitCode=0 Sep 09 23:19:30.728720 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.728402 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerDied","Data":"2e00b81a7c721432f17f72990b81d1e6beae0ab077087161b4d4fceb3ac2c589"} Sep 09 23:19:30.728720 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.728454 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerDied","Data":"ce87d4a62e4652e3ff69222eda32d4bd827fea650ec7fd1d0ee72f3920f48d8e"} Sep 09 23:19:30.742087 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.742068 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:30.779758 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.779733 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-cluster-tls-config\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.779911 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.779783 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb748af0-56d4-4d5b-8193-735bdbdb85bc-config-out\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.779911 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.779822 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-trusted-ca-bundle\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.779911 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.779851 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-main-tls\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.779911 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.779882 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-web-config\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.779911 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.779906 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.780395 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.779935 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.780395 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.779978 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-metrics-client-ca\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.780395 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.780012 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb748af0-56d4-4d5b-8193-735bdbdb85bc-tls-assets\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.780395 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.780050 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-main-db\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.780395 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.780074 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy-web\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.780395 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.780103 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdf67\" (UniqueName: \"kubernetes.io/projected/fb748af0-56d4-4d5b-8193-735bdbdb85bc-kube-api-access-bdf67\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.780395 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.780178 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-config-volume\") pod \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\" (UID: \"fb748af0-56d4-4d5b-8193-735bdbdb85bc\") " Sep 09 23:19:30.780749 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.780720 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:19:30.781336 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.781306 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:19:30.782725 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.782535 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:19:30.785486 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.785448 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb748af0-56d4-4d5b-8193-735bdbdb85bc-config-out" (OuterVolumeSpecName: "config-out") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:19:30.785486 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.785461 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:19:30.785637 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.785591 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:19:30.785810 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.785731 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb748af0-56d4-4d5b-8193-735bdbdb85bc-kube-api-access-bdf67" (OuterVolumeSpecName: "kube-api-access-bdf67") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "kube-api-access-bdf67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:19:30.786160 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.786005 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb748af0-56d4-4d5b-8193-735bdbdb85bc-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:19:30.786160 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.786077 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:19:30.788919 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.788717 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:19:30.788919 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.788836 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-config-volume" (OuterVolumeSpecName: "config-volume") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:19:30.790752 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.790726 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:19:30.796840 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.796822 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-web-config" (OuterVolumeSpecName: "web-config") pod "fb748af0-56d4-4d5b-8193-735bdbdb85bc" (UID: "fb748af0-56d4-4d5b-8193-735bdbdb85bc"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:19:30.881499 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881422 2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-config-volume\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:30.881499 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881448 2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-cluster-tls-config\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:30.881499 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881461 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb748af0-56d4-4d5b-8193-735bdbdb85bc-config-out\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:30.881499 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881475 2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:30.881499 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881491 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-main-tls\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:30.881743 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881505 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-web-config\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:30.881743 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881517 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:30.881743 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881530 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:30.881743 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881541 2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb748af0-56d4-4d5b-8193-735bdbdb85bc-metrics-client-ca\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:30.881743 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881553 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb748af0-56d4-4d5b-8193-735bdbdb85bc-tls-assets\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:30.881743 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881564 2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb748af0-56d4-4d5b-8193-735bdbdb85bc-alertmanager-main-db\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:30.881743 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881576 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb748af0-56d4-4d5b-8193-735bdbdb85bc-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:30.881743 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:30.881588 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdf67\" (UniqueName: \"kubernetes.io/projected/fb748af0-56d4-4d5b-8193-735bdbdb85bc-kube-api-access-bdf67\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:19:31.735187 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.735147 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb748af0-56d4-4d5b-8193-735bdbdb85bc","Type":"ContainerDied","Data":"a339c78d2bf6dec667fcdf34b2499ce0ceb5e1f7d223ea4c7b26658be5153922"} Sep 09 23:19:31.735606 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.735194 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.735606 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.735237 2580 scope.go:117] "RemoveContainer" containerID="0d4f8fd807d94ef56448d018cc46400da7aaabc599e6f94ebed2d22798effa59" Sep 09 23:19:31.742479 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.742462 2580 scope.go:117] "RemoveContainer" containerID="2e00b81a7c721432f17f72990b81d1e6beae0ab077087161b4d4fceb3ac2c589" Sep 09 23:19:31.748775 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.748755 2580 scope.go:117] "RemoveContainer" containerID="0f077b3f7d7e0a69b42ef94e6b1fbc020fbb82abc323121313a4685bfd5559e3" Sep 09 23:19:31.754833 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.754812 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 23:19:31.755337 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.755322 2580 scope.go:117] "RemoveContainer" containerID="ce87d4a62e4652e3ff69222eda32d4bd827fea650ec7fd1d0ee72f3920f48d8e" Sep 09 23:19:31.758331 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.758311 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 23:19:31.761504 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.761489 2580 scope.go:117] "RemoveContainer" containerID="05625f67b6dcd0f25f9066162c4c12df8cb8376ef810607b68bb6ad868db5baa" Sep 09 23:19:31.767422 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.767408 2580 scope.go:117] "RemoveContainer" containerID="a29c2ba94b1c5c9ec6c49c2623b5269b67c7bdce630ff2b16490cfd25923434b" Sep 09 23:19:31.773739 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.773715 2580 scope.go:117] "RemoveContainer" containerID="40f807149701b243df30aaaf374a51ec98bf4ebb318d8665275a389542acaba5" Sep 09 23:19:31.781732 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.781708 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 23:19:31.781922 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.781910 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="init-config-reloader" Sep 09 23:19:31.781992 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.781923 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="init-config-reloader" Sep 09 23:19:31.781992 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.781932 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="config-reloader" Sep 09 23:19:31.781992 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.781937 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="config-reloader" Sep 09 23:19:31.781992 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.781944 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="kube-rbac-proxy-metric" Sep 09 23:19:31.781992 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.781949 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="kube-rbac-proxy-metric" Sep 09 23:19:31.781992 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.781960 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="prom-label-proxy" Sep 09 23:19:31.781992 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.781978 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="prom-label-proxy" Sep 09 23:19:31.781992 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.781991 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68e3b494-ecda-49ca-a06d-5d941dea78c7" containerName="console" Sep 09 23:19:31.781992 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.781996 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="68e3b494-ecda-49ca-a06d-5d941dea78c7" containerName="console" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782002 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="kube-rbac-proxy-web" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782007 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="kube-rbac-proxy-web" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782013 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="kube-rbac-proxy" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782018 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="kube-rbac-proxy" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782026 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="alertmanager" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782032 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="alertmanager" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782069 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="alertmanager" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782076 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="kube-rbac-proxy-web" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782082 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="kube-rbac-proxy" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782087 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="kube-rbac-proxy-metric" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782092 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="prom-label-proxy" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782097 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="68e3b494-ecda-49ca-a06d-5d941dea78c7" containerName="console" Sep 09 23:19:31.782356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.782103 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" containerName="config-reloader" Sep 09 23:19:31.785220 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.785204 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.787506 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.787489 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 09 23:19:31.787599 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.787493 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 09 23:19:31.788078 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.788060 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 09 23:19:31.788078 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.788073 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 09 23:19:31.788198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.788160 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 09 23:19:31.788198 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.788164 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-l8fml\"" Sep 09 23:19:31.788355 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.788337 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 09 23:19:31.788489 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.788472 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 09 23:19:31.788644 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.788582 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 09 23:19:31.796554 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.796531 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 09 23:19:31.801242 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.801222 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 23:19:31.887917 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.887849 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd6fee9b-7f86-4020-9f6e-6964d50996b0-config-out\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.887917 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.887882 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.887917 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.887900 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-web-config\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.888150 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.887943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd6fee9b-7f86-4020-9f6e-6964d50996b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.888150 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.888044 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.888150 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.888092 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fd6fee9b-7f86-4020-9f6e-6964d50996b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.888150 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.888115 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.888150 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.888142 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.888325 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.888172 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd6fee9b-7f86-4020-9f6e-6964d50996b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.888325 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.888197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.888325 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.888214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mp7h\" (UniqueName: \"kubernetes.io/projected/fd6fee9b-7f86-4020-9f6e-6964d50996b0-kube-api-access-7mp7h\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.888325 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.888257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd6fee9b-7f86-4020-9f6e-6964d50996b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.888325 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.888284 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.988564 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988535 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.988709 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fd6fee9b-7f86-4020-9f6e-6964d50996b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.988709 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.988827 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.988827 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd6fee9b-7f86-4020-9f6e-6964d50996b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.988827 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.988827 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988772 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mp7h\" (UniqueName: \"kubernetes.io/projected/fd6fee9b-7f86-4020-9f6e-6964d50996b0-kube-api-access-7mp7h\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.988827 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988797 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd6fee9b-7f86-4020-9f6e-6964d50996b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.988827 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988823 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.989133 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd6fee9b-7f86-4020-9f6e-6964d50996b0-config-out\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.989133 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.989133 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988913 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-web-config\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.989133 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.988959 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd6fee9b-7f86-4020-9f6e-6964d50996b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.989133 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.989068 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fd6fee9b-7f86-4020-9f6e-6964d50996b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.989568 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.989540 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd6fee9b-7f86-4020-9f6e-6964d50996b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.989935 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.989911 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd6fee9b-7f86-4020-9f6e-6964d50996b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.991615 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.991586 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.991720 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.991618 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.991788 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.991744 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.991868 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.991843 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.992086 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.992064 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.992504 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.992478 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd6fee9b-7f86-4020-9f6e-6964d50996b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.992574 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.992511 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.992723 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.992706 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd6fee9b-7f86-4020-9f6e-6964d50996b0-web-config\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.993252 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.993233 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd6fee9b-7f86-4020-9f6e-6964d50996b0-config-out\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:31.997663 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:31.997645 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mp7h\" (UniqueName: \"kubernetes.io/projected/fd6fee9b-7f86-4020-9f6e-6964d50996b0-kube-api-access-7mp7h\") pod \"alertmanager-main-0\" (UID: \"fd6fee9b-7f86-4020-9f6e-6964d50996b0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:32.098069 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:32.098032 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 23:19:32.372493 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:32.372471 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 23:19:32.374420 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:19:32.374381 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd6fee9b_7f86_4020_9f6e_6964d50996b0.slice/crio-557af71e312c313ffba2d16695d01dfa5c0e871d8380476879eba762515581ab WatchSource:0}: Error finding container 557af71e312c313ffba2d16695d01dfa5c0e871d8380476879eba762515581ab: Status 404 returned error can't find the container with id 557af71e312c313ffba2d16695d01dfa5c0e871d8380476879eba762515581ab Sep 09 23:19:32.740396 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:32.740317 2580 generic.go:356] "Generic (PLEG): container finished" podID="fd6fee9b-7f86-4020-9f6e-6964d50996b0" containerID="7a81a3ce8df4b1c2a81bfb037c3fdcb44513d761a210bb8faa42f11c4d1e7828" exitCode=0 Sep 09 23:19:32.740808 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:32.740398 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fd6fee9b-7f86-4020-9f6e-6964d50996b0","Type":"ContainerDied","Data":"7a81a3ce8df4b1c2a81bfb037c3fdcb44513d761a210bb8faa42f11c4d1e7828"} Sep 09 23:19:32.740808 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:32.740473 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fd6fee9b-7f86-4020-9f6e-6964d50996b0","Type":"ContainerStarted","Data":"557af71e312c313ffba2d16695d01dfa5c0e871d8380476879eba762515581ab"} Sep 09 23:19:33.123557 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.123523 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb748af0-56d4-4d5b-8193-735bdbdb85bc" path="/var/lib/kubelet/pods/fb748af0-56d4-4d5b-8193-735bdbdb85bc/volumes" Sep 09 23:19:33.519572 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.519535 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5867bb85c5-rh944"] Sep 09 23:19:33.522568 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.522542 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.524960 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.524937 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-w2lwt\"" Sep 09 23:19:33.525098 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.524987 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 09 23:19:33.525098 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.525056 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 09 23:19:33.525248 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.525228 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 09 23:19:33.525383 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.525360 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 09 23:19:33.525428 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.525383 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 09 23:19:33.531045 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.531020 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 09 23:19:33.541364 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.541332 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5867bb85c5-rh944"] Sep 09 23:19:33.599294 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.599257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-telemeter-client-tls\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.599432 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.599299 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-metrics-client-ca\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.599432 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.599326 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-federate-client-tls\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.599565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.599439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.599565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.599474 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-serving-certs-ca-bundle\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.599565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.599495 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-secret-telemeter-client\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.599565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.599509 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9z8q\" (UniqueName: \"kubernetes.io/projected/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-kube-api-access-l9z8q\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.599565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.599533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.700640 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.700596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-federate-client-tls\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.700828 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.700660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.700828 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.700685 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-serving-certs-ca-bundle\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.700828 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.700702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-secret-telemeter-client\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.700828 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.700718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9z8q\" (UniqueName: \"kubernetes.io/projected/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-kube-api-access-l9z8q\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.700828 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.700743 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.700828 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.700778 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-telemeter-client-tls\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.700828 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.700813 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-metrics-client-ca\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.701579 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.701553 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-metrics-client-ca\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.701723 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.701693 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.701785 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.701553 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-serving-certs-ca-bundle\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.703896 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.703824 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-federate-client-tls\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.703896 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.703824 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-telemeter-client-tls\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.704106 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.704026 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.704479 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.704239 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-secret-telemeter-client\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.709536 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.709517 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9z8q\" (UniqueName: \"kubernetes.io/projected/ea6cf1f1-1b3d-429b-8328-c144abb4ce46-kube-api-access-l9z8q\") pod \"telemeter-client-5867bb85c5-rh944\" (UID: \"ea6cf1f1-1b3d-429b-8328-c144abb4ce46\") " pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:33.748449 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.748417 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fd6fee9b-7f86-4020-9f6e-6964d50996b0","Type":"ContainerStarted","Data":"0edfaff97c4de1592302a0383de62c1d55bbd0acfe7b0fed010afdbff8a28923"} Sep 09 23:19:33.748795 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.748495 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fd6fee9b-7f86-4020-9f6e-6964d50996b0","Type":"ContainerStarted","Data":"2901cf1a0b01bbd43711a528ee744fde9e58b17f2a1acec03e414c5d8ac6fd6a"} Sep 09 23:19:33.748795 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.748536 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fd6fee9b-7f86-4020-9f6e-6964d50996b0","Type":"ContainerStarted","Data":"88930912eb70779dbec0f99e0739f19025d7a6d6fc9515d573c7a3def77f7941"} Sep 09 23:19:33.748795 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.748570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fd6fee9b-7f86-4020-9f6e-6964d50996b0","Type":"ContainerStarted","Data":"76b693d5ea8cf135ac9e05cc0c05dcbf7394f6ce653af6d6f563e20ce32bcca5"} Sep 09 23:19:33.748795 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.748609 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fd6fee9b-7f86-4020-9f6e-6964d50996b0","Type":"ContainerStarted","Data":"3f2936570ae4129b3af051d09d602c25062f581d275d97fb0490b9b6ad5cb482"} Sep 09 23:19:33.748795 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.748644 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fd6fee9b-7f86-4020-9f6e-6964d50996b0","Type":"ContainerStarted","Data":"84d88ea60458e27f3647b5da66b3ed821d011dc4a55d1862270627d0c65568cf"} Sep 09 23:19:33.771741 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.771657 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.757914843 podStartE2EDuration="2.771642675s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.727832ms" isStatefulPod=false podCreationTimestamp="2026-09-09 23:19:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:19:33.7699902 +0000 UTC m=+135.196413504" watchObservedRunningTime="2026-09-09 23:19:33.771642675 +0000 UTC m=+135.198065981" Sep 09 23:19:33.834147 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:33.834116 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" Sep 09 23:19:34.114956 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:34.114920 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5867bb85c5-rh944"] Sep 09 23:19:34.117696 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:19:34.117670 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6cf1f1_1b3d_429b_8328_c144abb4ce46.slice/crio-8eaaf0127abf7c21ecc56deb65e99a1714c6fa0bab194ec32b043d02d0249f1b WatchSource:0}: Error finding container 8eaaf0127abf7c21ecc56deb65e99a1714c6fa0bab194ec32b043d02d0249f1b: Status 404 returned error can't find the container with id 8eaaf0127abf7c21ecc56deb65e99a1714c6fa0bab194ec32b043d02d0249f1b Sep 09 23:19:34.754789 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:34.754750 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" event={"ID":"ea6cf1f1-1b3d-429b-8328-c144abb4ce46","Type":"ContainerStarted","Data":"8eaaf0127abf7c21ecc56deb65e99a1714c6fa0bab194ec32b043d02d0249f1b"} Sep 09 23:19:36.766368 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.766328 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" event={"ID":"ea6cf1f1-1b3d-429b-8328-c144abb4ce46","Type":"ContainerStarted","Data":"28c7e88db7ad519cc03146e184d03936a040cb9f18de68c9f5357271322bba67"} Sep 09 23:19:36.766742 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.766395 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" event={"ID":"ea6cf1f1-1b3d-429b-8328-c144abb4ce46","Type":"ContainerStarted","Data":"0230e193280b9f7bef602b063d826cad827559c389458c7955083e7948aaeccb"} Sep 09 23:19:36.766742 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.766423 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" event={"ID":"ea6cf1f1-1b3d-429b-8328-c144abb4ce46","Type":"ContainerStarted","Data":"3282dcea48d30d6997fb80a1bd77e38ad0bea34224cc796bc7b29dc5b3ea5c14"} Sep 09 23:19:36.789338 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.789286 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5867bb85c5-rh944" podStartSLOduration=1.873639447 podStartE2EDuration="3.789270326s" totalImagesPullingTime="1.915630879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:19:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:19:36.788996311 +0000 UTC m=+138.215419618" watchObservedRunningTime="2026-09-09 23:19:36.789270326 +0000 UTC m=+138.215693633" Sep 09 23:19:36.885190 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.885163 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b49888cdf-jlwkl"] Sep 09 23:19:36.887378 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.887358 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:36.899621 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.899601 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b49888cdf-jlwkl"] Sep 09 23:19:36.926519 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.926493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-config\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:36.926610 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.926524 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xb6p\" (UniqueName: \"kubernetes.io/projected/04cbd4e1-fcb5-4475-9135-868cb9d2f614-kube-api-access-4xb6p\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:36.926610 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.926559 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-serving-cert\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:36.926610 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.926581 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-oauth-serving-cert\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:36.926610 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.926603 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-service-ca\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:36.926748 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.926620 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-oauth-config\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:36.926748 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:36.926670 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-trusted-ca-bundle\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.026941 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.026890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-config\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.026941 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.026920 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xb6p\" (UniqueName: \"kubernetes.io/projected/04cbd4e1-fcb5-4475-9135-868cb9d2f614-kube-api-access-4xb6p\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.027069 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.026962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-serving-cert\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.027069 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.027006 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-oauth-serving-cert\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.027069 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.027033 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-service-ca\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.027069 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.027065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-oauth-config\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.027233 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.027111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-trusted-ca-bundle\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.027709 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.027687 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-oauth-serving-cert\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.027803 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.027715 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-config\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.027871 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.027828 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-service-ca\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.028019 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.027996 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-trusted-ca-bundle\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.029479 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.029452 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-serving-cert\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.029553 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.029487 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-oauth-config\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.035073 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.035047 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xb6p\" (UniqueName: \"kubernetes.io/projected/04cbd4e1-fcb5-4475-9135-868cb9d2f614-kube-api-access-4xb6p\") pod \"console-b49888cdf-jlwkl\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.195283 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.195249 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:37.462255 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.462225 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b49888cdf-jlwkl"] Sep 09 23:19:37.465037 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:19:37.465013 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04cbd4e1_fcb5_4475_9135_868cb9d2f614.slice/crio-e4843a2af56c15b6ad0efa51c6e17d3c876f05551c2c0bb7efc23a7569d261c0 WatchSource:0}: Error finding container e4843a2af56c15b6ad0efa51c6e17d3c876f05551c2c0bb7efc23a7569d261c0: Status 404 returned error can't find the container with id e4843a2af56c15b6ad0efa51c6e17d3c876f05551c2c0bb7efc23a7569d261c0 Sep 09 23:19:37.772074 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.771960 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b49888cdf-jlwkl" event={"ID":"04cbd4e1-fcb5-4475-9135-868cb9d2f614","Type":"ContainerStarted","Data":"3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a"} Sep 09 23:19:37.772074 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.772031 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b49888cdf-jlwkl" event={"ID":"04cbd4e1-fcb5-4475-9135-868cb9d2f614","Type":"ContainerStarted","Data":"e4843a2af56c15b6ad0efa51c6e17d3c876f05551c2c0bb7efc23a7569d261c0"} Sep 09 23:19:37.789980 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:37.789922 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b49888cdf-jlwkl" podStartSLOduration=1.789902622 podStartE2EDuration="1.789902622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:19:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:19:37.788320642 +0000 UTC m=+139.214743972" watchObservedRunningTime="2026-09-09 23:19:37.789902622 +0000 UTC m=+139.216325930" Sep 09 23:19:47.195873 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:47.195832 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:47.196329 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:47.195912 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:47.200620 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:47.200596 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:47.811589 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:47.811560 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:19:47.854670 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:19:47.854638 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-759fc57545-68zh8"] Sep 09 23:20:12.873853 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:12.873761 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-759fc57545-68zh8" podUID="a6e79a72-f526-42a7-9507-38c57161b457" containerName="console" containerID="cri-o://84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6" gracePeriod=15 Sep 09 23:20:13.109957 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.109931 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:20:13.225371 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.225282 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6e79a72-f526-42a7-9507-38c57161b457-console-oauth-config\") pod \"a6e79a72-f526-42a7-9507-38c57161b457\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " Sep 09 23:20:13.225371 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.225335 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-trusted-ca-bundle\") pod \"a6e79a72-f526-42a7-9507-38c57161b457\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " Sep 09 23:20:13.225371 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.225359 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-oauth-serving-cert\") pod \"a6e79a72-f526-42a7-9507-38c57161b457\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " Sep 09 23:20:13.225653 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.225377 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6e79a72-f526-42a7-9507-38c57161b457-console-serving-cert\") pod \"a6e79a72-f526-42a7-9507-38c57161b457\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " Sep 09 23:20:13.225653 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.225393 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sghc\" (UniqueName: \"kubernetes.io/projected/a6e79a72-f526-42a7-9507-38c57161b457-kube-api-access-5sghc\") pod \"a6e79a72-f526-42a7-9507-38c57161b457\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " Sep 09 23:20:13.225653 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.225422 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-console-config\") pod \"a6e79a72-f526-42a7-9507-38c57161b457\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " Sep 09 23:20:13.225653 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.225446 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-service-ca\") pod \"a6e79a72-f526-42a7-9507-38c57161b457\" (UID: \"a6e79a72-f526-42a7-9507-38c57161b457\") " Sep 09 23:20:13.225916 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.225865 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-console-config" (OuterVolumeSpecName: "console-config") pod "a6e79a72-f526-42a7-9507-38c57161b457" (UID: "a6e79a72-f526-42a7-9507-38c57161b457"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:20:13.225916 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.225888 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a6e79a72-f526-42a7-9507-38c57161b457" (UID: "a6e79a72-f526-42a7-9507-38c57161b457"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:20:13.226103 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.225928 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-service-ca" (OuterVolumeSpecName: "service-ca") pod "a6e79a72-f526-42a7-9507-38c57161b457" (UID: "a6e79a72-f526-42a7-9507-38c57161b457"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:20:13.226103 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.226000 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a6e79a72-f526-42a7-9507-38c57161b457" (UID: "a6e79a72-f526-42a7-9507-38c57161b457"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:20:13.227642 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.227620 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6e79a72-f526-42a7-9507-38c57161b457-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a6e79a72-f526-42a7-9507-38c57161b457" (UID: "a6e79a72-f526-42a7-9507-38c57161b457"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:20:13.228007 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.227962 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6e79a72-f526-42a7-9507-38c57161b457-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a6e79a72-f526-42a7-9507-38c57161b457" (UID: "a6e79a72-f526-42a7-9507-38c57161b457"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:20:13.228064 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.228035 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e79a72-f526-42a7-9507-38c57161b457-kube-api-access-5sghc" (OuterVolumeSpecName: "kube-api-access-5sghc") pod "a6e79a72-f526-42a7-9507-38c57161b457" (UID: "a6e79a72-f526-42a7-9507-38c57161b457"). InnerVolumeSpecName "kube-api-access-5sghc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:20:13.326700 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.326666 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-trusted-ca-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:20:13.326700 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.326697 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-oauth-serving-cert\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:20:13.326700 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.326711 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6e79a72-f526-42a7-9507-38c57161b457-console-serving-cert\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:20:13.326929 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.326719 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sghc\" (UniqueName: \"kubernetes.io/projected/a6e79a72-f526-42a7-9507-38c57161b457-kube-api-access-5sghc\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:20:13.326929 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.326730 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-console-config\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:20:13.326929 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.326738 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6e79a72-f526-42a7-9507-38c57161b457-service-ca\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:20:13.326929 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.326745 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6e79a72-f526-42a7-9507-38c57161b457-console-oauth-config\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:20:13.899904 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.899864 2580 generic.go:356] "Generic (PLEG): container finished" podID="a6e79a72-f526-42a7-9507-38c57161b457" containerID="84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6" exitCode=0 Sep 09 23:20:13.900384 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.899917 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-759fc57545-68zh8" event={"ID":"a6e79a72-f526-42a7-9507-38c57161b457","Type":"ContainerDied","Data":"84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6"} Sep 09 23:20:13.900384 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.900015 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-759fc57545-68zh8" event={"ID":"a6e79a72-f526-42a7-9507-38c57161b457","Type":"ContainerDied","Data":"873419cb9e01fbb6eef09d05a2414ceea7424911aeed91b024cadc87d99cbe68"} Sep 09 23:20:13.900384 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.899931 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-759fc57545-68zh8" Sep 09 23:20:13.900384 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.900033 2580 scope.go:117] "RemoveContainer" containerID="84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6" Sep 09 23:20:13.908253 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.908234 2580 scope.go:117] "RemoveContainer" containerID="84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6" Sep 09 23:20:13.908607 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:20:13.908585 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6\": container with ID starting with 84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6 not found: ID does not exist" containerID="84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6" Sep 09 23:20:13.908686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.908613 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6"} err="failed to get container status \"84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6\": rpc error: code = NotFound desc = could not find container \"84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6\": container with ID starting with 84d23788c872ca48fce6b23fae66d8019b6b1cdc32df45d2042c2e524d22acb6 not found: ID does not exist" Sep 09 23:20:13.920956 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.920933 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-759fc57545-68zh8"] Sep 09 23:20:13.924584 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:13.924563 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-759fc57545-68zh8"] Sep 09 23:20:15.122669 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:15.122638 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6e79a72-f526-42a7-9507-38c57161b457" path="/var/lib/kubelet/pods/a6e79a72-f526-42a7-9507-38c57161b457/volumes" Sep 09 23:20:50.761953 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.761918 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c89f48688-wkqxd"] Sep 09 23:20:50.762499 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.762196 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6e79a72-f526-42a7-9507-38c57161b457" containerName="console" Sep 09 23:20:50.762499 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.762209 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e79a72-f526-42a7-9507-38c57161b457" containerName="console" Sep 09 23:20:50.762499 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.762262 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6e79a72-f526-42a7-9507-38c57161b457" containerName="console" Sep 09 23:20:50.763938 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.763922 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.776233 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.776211 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c89f48688-wkqxd"] Sep 09 23:20:50.793263 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.793234 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-config\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.793385 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.793274 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-oauth-serving-cert\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.793385 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.793290 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp8jw\" (UniqueName: \"kubernetes.io/projected/038ede4e-b6f5-4a6a-85e7-29f0e4319524-kube-api-access-sp8jw\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.793385 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.793358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-serving-cert\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.793484 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.793395 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-oauth-config\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.793484 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.793436 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-service-ca\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.793484 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.793455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-trusted-ca-bundle\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.894219 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.894169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-service-ca\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.894219 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.894226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-trusted-ca-bundle\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.894467 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.894255 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-config\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.894467 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.894296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-oauth-serving-cert\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.894467 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.894318 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp8jw\" (UniqueName: \"kubernetes.io/projected/038ede4e-b6f5-4a6a-85e7-29f0e4319524-kube-api-access-sp8jw\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.894467 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.894365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-serving-cert\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.894837 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.894398 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-oauth-config\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.895445 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.895410 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-config\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.895553 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.895497 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-trusted-ca-bundle\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.895610 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.895558 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-service-ca\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.895804 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.895784 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-oauth-serving-cert\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.897714 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.897687 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-oauth-config\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.901124 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.901099 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-serving-cert\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:50.902539 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:50.902519 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp8jw\" (UniqueName: \"kubernetes.io/projected/038ede4e-b6f5-4a6a-85e7-29f0e4319524-kube-api-access-sp8jw\") pod \"console-6c89f48688-wkqxd\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:51.072211 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:51.072173 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:20:51.338504 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:51.338480 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c89f48688-wkqxd"] Sep 09 23:20:51.340501 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:20:51.340470 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod038ede4e_b6f5_4a6a_85e7_29f0e4319524.slice/crio-70f0e4251a5db493b3a51dd42a6ece4d67db38eedc1a08920bdee5bea7514496 WatchSource:0}: Error finding container 70f0e4251a5db493b3a51dd42a6ece4d67db38eedc1a08920bdee5bea7514496: Status 404 returned error can't find the container with id 70f0e4251a5db493b3a51dd42a6ece4d67db38eedc1a08920bdee5bea7514496 Sep 09 23:20:52.034946 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:52.034902 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c89f48688-wkqxd" event={"ID":"038ede4e-b6f5-4a6a-85e7-29f0e4319524","Type":"ContainerStarted","Data":"b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed"} Sep 09 23:20:52.035339 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:52.034959 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c89f48688-wkqxd" event={"ID":"038ede4e-b6f5-4a6a-85e7-29f0e4319524","Type":"ContainerStarted","Data":"70f0e4251a5db493b3a51dd42a6ece4d67db38eedc1a08920bdee5bea7514496"} Sep 09 23:20:52.052413 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:20:52.052362 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c89f48688-wkqxd" podStartSLOduration=2.052345892 podStartE2EDuration="2.052345892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:20:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:20:52.050677625 +0000 UTC m=+213.477100941" watchObservedRunningTime="2026-09-09 23:20:52.052345892 +0000 UTC m=+213.478769199" Sep 09 23:21:01.072771 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:01.072737 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:21:01.073239 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:01.072800 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:21:01.077314 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:01.077294 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:21:02.076609 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:02.076579 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:21:02.119410 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:02.119379 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b49888cdf-jlwkl"] Sep 09 23:21:27.139742 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.139693 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b49888cdf-jlwkl" podUID="04cbd4e1-fcb5-4475-9135-868cb9d2f614" containerName="console" containerID="cri-o://3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a" gracePeriod=15 Sep 09 23:21:27.378537 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.378511 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:21:27.470384 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.470297 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-oauth-serving-cert\") pod \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " Sep 09 23:21:27.470384 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.470356 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-service-ca\") pod \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " Sep 09 23:21:27.470384 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.470377 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-oauth-config\") pod \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " Sep 09 23:21:27.470641 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.470517 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-trusted-ca-bundle\") pod \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " Sep 09 23:21:27.470641 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.470591 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xb6p\" (UniqueName: \"kubernetes.io/projected/04cbd4e1-fcb5-4475-9135-868cb9d2f614-kube-api-access-4xb6p\") pod \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " Sep 09 23:21:27.470711 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.470641 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-serving-cert\") pod \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " Sep 09 23:21:27.470711 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.470691 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-service-ca" (OuterVolumeSpecName: "service-ca") pod "04cbd4e1-fcb5-4475-9135-868cb9d2f614" (UID: "04cbd4e1-fcb5-4475-9135-868cb9d2f614"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:21:27.470806 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.470783 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-config\") pod \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\" (UID: \"04cbd4e1-fcb5-4475-9135-868cb9d2f614\") " Sep 09 23:21:27.470859 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.470827 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "04cbd4e1-fcb5-4475-9135-868cb9d2f614" (UID: "04cbd4e1-fcb5-4475-9135-868cb9d2f614"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:21:27.470940 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.470917 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "04cbd4e1-fcb5-4475-9135-868cb9d2f614" (UID: "04cbd4e1-fcb5-4475-9135-868cb9d2f614"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:21:27.471238 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.471203 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-oauth-serving-cert\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:21:27.471238 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.471226 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-service-ca\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:21:27.471238 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.471223 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-config" (OuterVolumeSpecName: "console-config") pod "04cbd4e1-fcb5-4475-9135-868cb9d2f614" (UID: "04cbd4e1-fcb5-4475-9135-868cb9d2f614"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:21:27.471238 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.471240 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-trusted-ca-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:21:27.472621 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.472598 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "04cbd4e1-fcb5-4475-9135-868cb9d2f614" (UID: "04cbd4e1-fcb5-4475-9135-868cb9d2f614"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:21:27.472693 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.472660 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "04cbd4e1-fcb5-4475-9135-868cb9d2f614" (UID: "04cbd4e1-fcb5-4475-9135-868cb9d2f614"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:21:27.472693 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.472664 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04cbd4e1-fcb5-4475-9135-868cb9d2f614-kube-api-access-4xb6p" (OuterVolumeSpecName: "kube-api-access-4xb6p") pod "04cbd4e1-fcb5-4475-9135-868cb9d2f614" (UID: "04cbd4e1-fcb5-4475-9135-868cb9d2f614"). InnerVolumeSpecName "kube-api-access-4xb6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:21:27.572230 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.572184 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-oauth-config\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:21:27.572230 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.572226 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xb6p\" (UniqueName: \"kubernetes.io/projected/04cbd4e1-fcb5-4475-9135-868cb9d2f614-kube-api-access-4xb6p\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:21:27.572230 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.572236 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-serving-cert\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:21:27.572230 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:27.572245 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04cbd4e1-fcb5-4475-9135-868cb9d2f614-console-config\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:21:28.174512 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:28.174479 2580 generic.go:356] "Generic (PLEG): container finished" podID="04cbd4e1-fcb5-4475-9135-868cb9d2f614" containerID="3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a" exitCode=0 Sep 09 23:21:28.174904 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:28.174528 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b49888cdf-jlwkl" Sep 09 23:21:28.174904 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:28.174537 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b49888cdf-jlwkl" event={"ID":"04cbd4e1-fcb5-4475-9135-868cb9d2f614","Type":"ContainerDied","Data":"3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a"} Sep 09 23:21:28.174904 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:28.174597 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b49888cdf-jlwkl" event={"ID":"04cbd4e1-fcb5-4475-9135-868cb9d2f614","Type":"ContainerDied","Data":"e4843a2af56c15b6ad0efa51c6e17d3c876f05551c2c0bb7efc23a7569d261c0"} Sep 09 23:21:28.174904 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:28.174616 2580 scope.go:117] "RemoveContainer" containerID="3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a" Sep 09 23:21:28.182819 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:28.182804 2580 scope.go:117] "RemoveContainer" containerID="3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a" Sep 09 23:21:28.183080 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:21:28.183065 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a\": container with ID starting with 3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a not found: ID does not exist" containerID="3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a" Sep 09 23:21:28.183124 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:28.183087 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a"} err="failed to get container status \"3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a\": rpc error: code = NotFound desc = could not find container \"3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a\": container with ID starting with 3caf2df06e5a6041cd183691878d8cffd8a11c0005971ba28943dbe4f633b22a not found: ID does not exist" Sep 09 23:21:28.193732 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:28.193710 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b49888cdf-jlwkl"] Sep 09 23:21:28.195786 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:28.195765 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b49888cdf-jlwkl"] Sep 09 23:21:29.122378 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:29.122346 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04cbd4e1-fcb5-4475-9135-868cb9d2f614" path="/var/lib/kubelet/pods/04cbd4e1-fcb5-4475-9135-868cb9d2f614/volumes" Sep 09 23:21:35.764791 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:35.764756 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq"] Sep 09 23:21:35.765183 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:35.765049 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04cbd4e1-fcb5-4475-9135-868cb9d2f614" containerName="console" Sep 09 23:21:35.765183 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:35.765061 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="04cbd4e1-fcb5-4475-9135-868cb9d2f614" containerName="console" Sep 09 23:21:35.765183 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:35.765111 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="04cbd4e1-fcb5-4475-9135-868cb9d2f614" containerName="console" Sep 09 23:21:35.768894 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:35.768876 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:35.771410 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:35.771387 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 09 23:21:35.771546 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:35.771431 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 09 23:21:35.771546 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:35.771501 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bkm2k\"" Sep 09 23:21:35.776377 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:35.776352 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq"] Sep 09 23:21:35.936134 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:35.936081 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq\" (UID: \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:35.936134 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:35.936135 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2glhm\" (UniqueName: \"kubernetes.io/projected/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-kube-api-access-2glhm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq\" (UID: \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:35.936361 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:35.936233 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq\" (UID: \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:36.037388 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:36.037358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq\" (UID: \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:36.037529 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:36.037400 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2glhm\" (UniqueName: \"kubernetes.io/projected/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-kube-api-access-2glhm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq\" (UID: \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:36.037529 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:36.037434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq\" (UID: \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:36.037759 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:36.037737 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq\" (UID: \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:36.037817 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:36.037766 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq\" (UID: \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:36.046154 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:36.046132 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2glhm\" (UniqueName: \"kubernetes.io/projected/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-kube-api-access-2glhm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq\" (UID: \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:36.078211 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:36.078186 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:36.344325 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:36.344300 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq"] Sep 09 23:21:36.347102 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:21:36.347077 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29c7b045_f8cf_41bd_ae2a_a79e25c1fa40.slice/crio-0d41b8779c720d8a2b652cf546fdc16b06f81afe689f7f1f65a7b2ff97b45aa5 WatchSource:0}: Error finding container 0d41b8779c720d8a2b652cf546fdc16b06f81afe689f7f1f65a7b2ff97b45aa5: Status 404 returned error can't find the container with id 0d41b8779c720d8a2b652cf546fdc16b06f81afe689f7f1f65a7b2ff97b45aa5 Sep 09 23:21:37.208789 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:37.208751 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" event={"ID":"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40","Type":"ContainerStarted","Data":"0d41b8779c720d8a2b652cf546fdc16b06f81afe689f7f1f65a7b2ff97b45aa5"} Sep 09 23:21:43.232317 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:43.232280 2580 generic.go:356] "Generic (PLEG): container finished" podID="29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" containerID="65180dc092b233df8d3c9b0a10083929521e2f1d4f7fdd91636db4082892a40f" exitCode=0 Sep 09 23:21:43.232792 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:43.232338 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" event={"ID":"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40","Type":"ContainerDied","Data":"65180dc092b233df8d3c9b0a10083929521e2f1d4f7fdd91636db4082892a40f"} Sep 09 23:21:46.248980 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:46.248936 2580 generic.go:356] "Generic (PLEG): container finished" podID="29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" containerID="2a1a9ea95bb0bd0655f0b7b7e57ca541278bb57176e539085a68f809fe8a77f1" exitCode=0 Sep 09 23:21:46.249360 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:46.249001 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" event={"ID":"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40","Type":"ContainerDied","Data":"2a1a9ea95bb0bd0655f0b7b7e57ca541278bb57176e539085a68f809fe8a77f1"} Sep 09 23:21:53.281329 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:53.281239 2580 generic.go:356] "Generic (PLEG): container finished" podID="29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" containerID="52f7711017429753fd2d69627e150cbbf9b07bfb730ad2350d339603963a55af" exitCode=0 Sep 09 23:21:53.281329 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:53.281310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" event={"ID":"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40","Type":"ContainerDied","Data":"52f7711017429753fd2d69627e150cbbf9b07bfb730ad2350d339603963a55af"} Sep 09 23:21:54.408217 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:54.408192 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:54.491236 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:54.491209 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-bundle\") pod \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\" (UID: \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\") " Sep 09 23:21:54.491378 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:54.491269 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2glhm\" (UniqueName: \"kubernetes.io/projected/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-kube-api-access-2glhm\") pod \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\" (UID: \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\") " Sep 09 23:21:54.491378 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:54.491299 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-util\") pod \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\" (UID: \"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40\") " Sep 09 23:21:54.491862 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:54.491835 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-bundle" (OuterVolumeSpecName: "bundle") pod "29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" (UID: "29c7b045-f8cf-41bd-ae2a-a79e25c1fa40"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:21:54.493378 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:54.493355 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-kube-api-access-2glhm" (OuterVolumeSpecName: "kube-api-access-2glhm") pod "29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" (UID: "29c7b045-f8cf-41bd-ae2a-a79e25c1fa40"). InnerVolumeSpecName "kube-api-access-2glhm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:21:54.495955 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:54.495932 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-util" (OuterVolumeSpecName: "util") pod "29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" (UID: "29c7b045-f8cf-41bd-ae2a-a79e25c1fa40"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:21:54.591988 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:54.591885 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-util\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:21:54.591988 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:54.591909 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:21:54.591988 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:54.591919 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2glhm\" (UniqueName: \"kubernetes.io/projected/29c7b045-f8cf-41bd-ae2a-a79e25c1fa40-kube-api-access-2glhm\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:21:55.294904 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:55.294869 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" event={"ID":"29c7b045-f8cf-41bd-ae2a-a79e25c1fa40","Type":"ContainerDied","Data":"0d41b8779c720d8a2b652cf546fdc16b06f81afe689f7f1f65a7b2ff97b45aa5"} Sep 09 23:21:55.294904 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:55.294893 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pqrfq" Sep 09 23:21:55.295074 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:55.294918 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d41b8779c720d8a2b652cf546fdc16b06f81afe689f7f1f65a7b2ff97b45aa5" Sep 09 23:21:58.511999 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.511948 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz"] Sep 09 23:21:58.512498 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.512219 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" containerName="extract" Sep 09 23:21:58.512498 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.512230 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" containerName="extract" Sep 09 23:21:58.512498 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.512249 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" containerName="util" Sep 09 23:21:58.512498 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.512254 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" containerName="util" Sep 09 23:21:58.512498 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.512264 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" containerName="pull" Sep 09 23:21:58.512498 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.512269 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" containerName="pull" Sep 09 23:21:58.512498 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.512309 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29c7b045-f8cf-41bd-ae2a-a79e25c1fa40" containerName="extract" Sep 09 23:21:58.545794 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.545755 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz"] Sep 09 23:21:58.546008 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.545896 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz" Sep 09 23:21:58.548905 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.548877 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ww529\"" Sep 09 23:21:58.549069 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.548921 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 09 23:21:58.549069 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.548921 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 09 23:21:58.620286 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.620247 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kdrf\" (UniqueName: \"kubernetes.io/projected/eed7582d-fca0-4ffd-a951-02b97709763a-kube-api-access-9kdrf\") pod \"cert-manager-operator-controller-manager-55c444bf86-x8pkz\" (UID: \"eed7582d-fca0-4ffd-a951-02b97709763a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz" Sep 09 23:21:58.620286 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.620288 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eed7582d-fca0-4ffd-a951-02b97709763a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x8pkz\" (UID: \"eed7582d-fca0-4ffd-a951-02b97709763a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz" Sep 09 23:21:58.721504 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.721479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kdrf\" (UniqueName: \"kubernetes.io/projected/eed7582d-fca0-4ffd-a951-02b97709763a-kube-api-access-9kdrf\") pod \"cert-manager-operator-controller-manager-55c444bf86-x8pkz\" (UID: \"eed7582d-fca0-4ffd-a951-02b97709763a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz" Sep 09 23:21:58.721590 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.721522 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eed7582d-fca0-4ffd-a951-02b97709763a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x8pkz\" (UID: \"eed7582d-fca0-4ffd-a951-02b97709763a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz" Sep 09 23:21:58.721867 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.721851 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eed7582d-fca0-4ffd-a951-02b97709763a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x8pkz\" (UID: \"eed7582d-fca0-4ffd-a951-02b97709763a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz" Sep 09 23:21:58.729919 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.729886 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kdrf\" (UniqueName: \"kubernetes.io/projected/eed7582d-fca0-4ffd-a951-02b97709763a-kube-api-access-9kdrf\") pod \"cert-manager-operator-controller-manager-55c444bf86-x8pkz\" (UID: \"eed7582d-fca0-4ffd-a951-02b97709763a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz" Sep 09 23:21:58.854757 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:58.854715 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz" Sep 09 23:21:59.159594 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:59.159566 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz"] Sep 09 23:21:59.160954 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:21:59.160927 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed7582d_fca0_4ffd_a951_02b97709763a.slice/crio-bf4b7024980320a1017b7f7f57301292cc2acbffc0162d3ed0b0e38fc84c8a41 WatchSource:0}: Error finding container bf4b7024980320a1017b7f7f57301292cc2acbffc0162d3ed0b0e38fc84c8a41: Status 404 returned error can't find the container with id bf4b7024980320a1017b7f7f57301292cc2acbffc0162d3ed0b0e38fc84c8a41 Sep 09 23:21:59.313960 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:21:59.313891 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz" event={"ID":"eed7582d-fca0-4ffd-a951-02b97709763a","Type":"ContainerStarted","Data":"bf4b7024980320a1017b7f7f57301292cc2acbffc0162d3ed0b0e38fc84c8a41"} Sep 09 23:22:02.329583 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:02.329544 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz" event={"ID":"eed7582d-fca0-4ffd-a951-02b97709763a","Type":"ContainerStarted","Data":"62ea963e19e7685ea1bf7aa5c23c379331a4357c47c0da985c1940fe0987cdcd"} Sep 09 23:22:02.357860 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:02.357806 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8pkz" podStartSLOduration=2.094685268 podStartE2EDuration="4.357788726s" totalImagesPullingTime="2.263103458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:22:02.356638668 +0000 UTC m=+283.783061977" watchObservedRunningTime="2026-09-09 23:22:02.357788726 +0000 UTC m=+283.784212033" Sep 09 23:22:03.236105 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.236071 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr"] Sep 09 23:22:03.261810 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.261782 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr"] Sep 09 23:22:03.261997 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.261903 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:03.264600 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.264578 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 09 23:22:03.265564 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.265547 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bkm2k\"" Sep 09 23:22:03.265658 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.265552 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 09 23:22:03.357468 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.357434 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p796\" (UniqueName: \"kubernetes.io/projected/24d94c22-6600-483e-9562-549707afa24d-kube-api-access-7p796\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr\" (UID: \"24d94c22-6600-483e-9562-549707afa24d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:03.357468 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.357471 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24d94c22-6600-483e-9562-549707afa24d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr\" (UID: \"24d94c22-6600-483e-9562-549707afa24d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:03.357869 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.357493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24d94c22-6600-483e-9562-549707afa24d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr\" (UID: \"24d94c22-6600-483e-9562-549707afa24d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:03.458049 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.458012 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p796\" (UniqueName: \"kubernetes.io/projected/24d94c22-6600-483e-9562-549707afa24d-kube-api-access-7p796\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr\" (UID: \"24d94c22-6600-483e-9562-549707afa24d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:03.458049 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.458055 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24d94c22-6600-483e-9562-549707afa24d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr\" (UID: \"24d94c22-6600-483e-9562-549707afa24d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:03.458233 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.458078 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24d94c22-6600-483e-9562-549707afa24d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr\" (UID: \"24d94c22-6600-483e-9562-549707afa24d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:03.458443 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.458418 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24d94c22-6600-483e-9562-549707afa24d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr\" (UID: \"24d94c22-6600-483e-9562-549707afa24d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:03.458482 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.458427 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24d94c22-6600-483e-9562-549707afa24d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr\" (UID: \"24d94c22-6600-483e-9562-549707afa24d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:03.467316 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.467284 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p796\" (UniqueName: \"kubernetes.io/projected/24d94c22-6600-483e-9562-549707afa24d-kube-api-access-7p796\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr\" (UID: \"24d94c22-6600-483e-9562-549707afa24d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:03.571096 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.571068 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:03.855770 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:03.855746 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr"] Sep 09 23:22:03.857673 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:22:03.857645 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d94c22_6600_483e_9562_549707afa24d.slice/crio-010ddbbf6643c1c4ca6afbd642bb7d29c074bd7ea7ac067cd594b1a3e5de279a WatchSource:0}: Error finding container 010ddbbf6643c1c4ca6afbd642bb7d29c074bd7ea7ac067cd594b1a3e5de279a: Status 404 returned error can't find the container with id 010ddbbf6643c1c4ca6afbd642bb7d29c074bd7ea7ac067cd594b1a3e5de279a Sep 09 23:22:04.338250 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:04.338213 2580 generic.go:356] "Generic (PLEG): container finished" podID="24d94c22-6600-483e-9562-549707afa24d" containerID="8b85406715ecf9dcc2f851222fbcc6c70b3eef0ed7882547caf832db7da68e8f" exitCode=0 Sep 09 23:22:04.338425 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:04.338299 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" event={"ID":"24d94c22-6600-483e-9562-549707afa24d","Type":"ContainerDied","Data":"8b85406715ecf9dcc2f851222fbcc6c70b3eef0ed7882547caf832db7da68e8f"} Sep 09 23:22:04.338425 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:04.338353 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" event={"ID":"24d94c22-6600-483e-9562-549707afa24d","Type":"ContainerStarted","Data":"010ddbbf6643c1c4ca6afbd642bb7d29c074bd7ea7ac067cd594b1a3e5de279a"} Sep 09 23:22:05.138147 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.138102 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4"] Sep 09 23:22:05.158122 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.158085 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4"] Sep 09 23:22:05.158282 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.158180 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4" Sep 09 23:22:05.161026 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.160998 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8k747\"" Sep 09 23:22:05.161026 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.161014 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 09 23:22:05.161250 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.161057 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 09 23:22:05.273919 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.273877 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmxps\" (UniqueName: \"kubernetes.io/projected/45478d8e-75e9-4e2d-9ad6-3418cd17cf9e-kube-api-access-rmxps\") pod \"cert-manager-cainjector-559cdd8c87-4w5f4\" (UID: \"45478d8e-75e9-4e2d-9ad6-3418cd17cf9e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4" Sep 09 23:22:05.274113 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.273934 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45478d8e-75e9-4e2d-9ad6-3418cd17cf9e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4w5f4\" (UID: \"45478d8e-75e9-4e2d-9ad6-3418cd17cf9e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4" Sep 09 23:22:05.375138 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.375091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxps\" (UniqueName: \"kubernetes.io/projected/45478d8e-75e9-4e2d-9ad6-3418cd17cf9e-kube-api-access-rmxps\") pod \"cert-manager-cainjector-559cdd8c87-4w5f4\" (UID: \"45478d8e-75e9-4e2d-9ad6-3418cd17cf9e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4" Sep 09 23:22:05.375323 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.375178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45478d8e-75e9-4e2d-9ad6-3418cd17cf9e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4w5f4\" (UID: \"45478d8e-75e9-4e2d-9ad6-3418cd17cf9e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4" Sep 09 23:22:05.383715 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.383685 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45478d8e-75e9-4e2d-9ad6-3418cd17cf9e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4w5f4\" (UID: \"45478d8e-75e9-4e2d-9ad6-3418cd17cf9e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4" Sep 09 23:22:05.383834 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.383807 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmxps\" (UniqueName: \"kubernetes.io/projected/45478d8e-75e9-4e2d-9ad6-3418cd17cf9e-kube-api-access-rmxps\") pod \"cert-manager-cainjector-559cdd8c87-4w5f4\" (UID: \"45478d8e-75e9-4e2d-9ad6-3418cd17cf9e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4" Sep 09 23:22:05.479550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.479468 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4" Sep 09 23:22:05.771071 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:05.771046 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4"] Sep 09 23:22:05.774143 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:22:05.774109 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45478d8e_75e9_4e2d_9ad6_3418cd17cf9e.slice/crio-9cf6e7f305be3c43fd0e400fae87466acedc68d10936dd1c9cfa7529ac7d93bd WatchSource:0}: Error finding container 9cf6e7f305be3c43fd0e400fae87466acedc68d10936dd1c9cfa7529ac7d93bd: Status 404 returned error can't find the container with id 9cf6e7f305be3c43fd0e400fae87466acedc68d10936dd1c9cfa7529ac7d93bd Sep 09 23:22:06.348561 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:06.348528 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4" event={"ID":"45478d8e-75e9-4e2d-9ad6-3418cd17cf9e","Type":"ContainerStarted","Data":"9cf6e7f305be3c43fd0e400fae87466acedc68d10936dd1c9cfa7529ac7d93bd"} Sep 09 23:22:07.355113 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:07.355076 2580 generic.go:356] "Generic (PLEG): container finished" podID="24d94c22-6600-483e-9562-549707afa24d" containerID="5cb1ed65aae5ab9efc33f6771f5f76fc5db1076dca8c06e4e1100a299e6eb4c7" exitCode=0 Sep 09 23:22:07.355535 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:07.355149 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" event={"ID":"24d94c22-6600-483e-9562-549707afa24d","Type":"ContainerDied","Data":"5cb1ed65aae5ab9efc33f6771f5f76fc5db1076dca8c06e4e1100a299e6eb4c7"} Sep 09 23:22:08.363506 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:08.363470 2580 generic.go:356] "Generic (PLEG): container finished" podID="24d94c22-6600-483e-9562-549707afa24d" containerID="9273c7f347ae059366d3cc0fef7ec61c118cba024d7d921ee2b6285dc2bfc923" exitCode=0 Sep 09 23:22:08.363938 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:08.363559 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" event={"ID":"24d94c22-6600-483e-9562-549707afa24d","Type":"ContainerDied","Data":"9273c7f347ae059366d3cc0fef7ec61c118cba024d7d921ee2b6285dc2bfc923"} Sep 09 23:22:09.334171 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.334131 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w56w9"] Sep 09 23:22:09.336308 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.336291 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" Sep 09 23:22:09.339124 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.339105 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-rplxj\"" Sep 09 23:22:09.345347 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.345322 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w56w9"] Sep 09 23:22:09.369553 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.369526 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4" event={"ID":"45478d8e-75e9-4e2d-9ad6-3418cd17cf9e","Type":"ContainerStarted","Data":"dba7f8a9171ca1fdef5cdad7d0bb862fdaf997a48dc5bd380a448851ab2bfd24"} Sep 09 23:22:09.387079 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.387032 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4w5f4" podStartSLOduration=1.4379407880000001 podStartE2EDuration="4.387016546s" totalImagesPullingTime="2.949075758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:22:09.385723078 +0000 UTC m=+290.812146385" watchObservedRunningTime="2026-09-09 23:22:09.387016546 +0000 UTC m=+290.813439852" Sep 09 23:22:09.411736 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.411694 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4pw7\" (UniqueName: \"kubernetes.io/projected/21efc05f-e39d-43e4-975a-45998ddc2f16-kube-api-access-s4pw7\") pod \"cert-manager-webhook-6486ffd997-w56w9\" (UID: \"21efc05f-e39d-43e4-975a-45998ddc2f16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" Sep 09 23:22:09.411736 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.411734 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21efc05f-e39d-43e4-975a-45998ddc2f16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w56w9\" (UID: \"21efc05f-e39d-43e4-975a-45998ddc2f16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" Sep 09 23:22:09.492167 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.492138 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:09.513417 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.513384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4pw7\" (UniqueName: \"kubernetes.io/projected/21efc05f-e39d-43e4-975a-45998ddc2f16-kube-api-access-s4pw7\") pod \"cert-manager-webhook-6486ffd997-w56w9\" (UID: \"21efc05f-e39d-43e4-975a-45998ddc2f16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" Sep 09 23:22:09.513417 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.513417 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21efc05f-e39d-43e4-975a-45998ddc2f16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w56w9\" (UID: \"21efc05f-e39d-43e4-975a-45998ddc2f16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" Sep 09 23:22:09.520866 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.520833 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21efc05f-e39d-43e4-975a-45998ddc2f16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w56w9\" (UID: \"21efc05f-e39d-43e4-975a-45998ddc2f16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" Sep 09 23:22:09.520993 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.520930 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4pw7\" (UniqueName: \"kubernetes.io/projected/21efc05f-e39d-43e4-975a-45998ddc2f16-kube-api-access-s4pw7\") pod \"cert-manager-webhook-6486ffd997-w56w9\" (UID: \"21efc05f-e39d-43e4-975a-45998ddc2f16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" Sep 09 23:22:09.613832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.613730 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24d94c22-6600-483e-9562-549707afa24d-bundle\") pod \"24d94c22-6600-483e-9562-549707afa24d\" (UID: \"24d94c22-6600-483e-9562-549707afa24d\") " Sep 09 23:22:09.613832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.613794 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24d94c22-6600-483e-9562-549707afa24d-util\") pod \"24d94c22-6600-483e-9562-549707afa24d\" (UID: \"24d94c22-6600-483e-9562-549707afa24d\") " Sep 09 23:22:09.613832 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.613834 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p796\" (UniqueName: \"kubernetes.io/projected/24d94c22-6600-483e-9562-549707afa24d-kube-api-access-7p796\") pod \"24d94c22-6600-483e-9562-549707afa24d\" (UID: \"24d94c22-6600-483e-9562-549707afa24d\") " Sep 09 23:22:09.614246 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.614218 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24d94c22-6600-483e-9562-549707afa24d-bundle" (OuterVolumeSpecName: "bundle") pod "24d94c22-6600-483e-9562-549707afa24d" (UID: "24d94c22-6600-483e-9562-549707afa24d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:22:09.615924 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.615901 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24d94c22-6600-483e-9562-549707afa24d-kube-api-access-7p796" (OuterVolumeSpecName: "kube-api-access-7p796") pod "24d94c22-6600-483e-9562-549707afa24d" (UID: "24d94c22-6600-483e-9562-549707afa24d"). InnerVolumeSpecName "kube-api-access-7p796". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:22:09.618234 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.618209 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24d94c22-6600-483e-9562-549707afa24d-util" (OuterVolumeSpecName: "util") pod "24d94c22-6600-483e-9562-549707afa24d" (UID: "24d94c22-6600-483e-9562-549707afa24d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:22:09.647095 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.647068 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" Sep 09 23:22:09.715654 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.715626 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p796\" (UniqueName: \"kubernetes.io/projected/24d94c22-6600-483e-9562-549707afa24d-kube-api-access-7p796\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:22:09.715792 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.715675 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24d94c22-6600-483e-9562-549707afa24d-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:22:09.715792 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.715754 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24d94c22-6600-483e-9562-549707afa24d-util\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:22:09.912651 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:09.912577 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w56w9"] Sep 09 23:22:09.915335 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:22:09.915298 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21efc05f_e39d_43e4_975a_45998ddc2f16.slice/crio-26b48a0d8f128a9b4868f9d891814bafe0faaa87418e128051f2cf3bcae5e9f8 WatchSource:0}: Error finding container 26b48a0d8f128a9b4868f9d891814bafe0faaa87418e128051f2cf3bcae5e9f8: Status 404 returned error can't find the container with id 26b48a0d8f128a9b4868f9d891814bafe0faaa87418e128051f2cf3bcae5e9f8 Sep 09 23:22:10.375849 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:10.375810 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" event={"ID":"24d94c22-6600-483e-9562-549707afa24d","Type":"ContainerDied","Data":"010ddbbf6643c1c4ca6afbd642bb7d29c074bd7ea7ac067cd594b1a3e5de279a"} Sep 09 23:22:10.376335 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:10.375830 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fk47hr" Sep 09 23:22:10.376335 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:10.375869 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="010ddbbf6643c1c4ca6afbd642bb7d29c074bd7ea7ac067cd594b1a3e5de279a" Sep 09 23:22:10.377739 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:10.377714 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" event={"ID":"21efc05f-e39d-43e4-975a-45998ddc2f16","Type":"ContainerStarted","Data":"051f122b87ffa34d0cf58531d1961852c7d05873ff8f36e9883349e9758c8c62"} Sep 09 23:22:10.377862 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:10.377751 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" event={"ID":"21efc05f-e39d-43e4-975a-45998ddc2f16","Type":"ContainerStarted","Data":"26b48a0d8f128a9b4868f9d891814bafe0faaa87418e128051f2cf3bcae5e9f8"} Sep 09 23:22:10.377922 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:10.377906 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" Sep 09 23:22:10.393304 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:10.393255 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" podStartSLOduration=1.3932376 podStartE2EDuration="1.3932376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:22:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:22:10.39283958 +0000 UTC m=+291.819262888" watchObservedRunningTime="2026-09-09 23:22:10.3932376 +0000 UTC m=+291.819660907" Sep 09 23:22:16.383366 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.383331 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-w56w9" Sep 09 23:22:16.802950 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.802920 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x"] Sep 09 23:22:16.803222 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.803210 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24d94c22-6600-483e-9562-549707afa24d" containerName="extract" Sep 09 23:22:16.803275 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.803224 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="24d94c22-6600-483e-9562-549707afa24d" containerName="extract" Sep 09 23:22:16.803275 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.803241 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24d94c22-6600-483e-9562-549707afa24d" containerName="pull" Sep 09 23:22:16.803275 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.803246 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="24d94c22-6600-483e-9562-549707afa24d" containerName="pull" Sep 09 23:22:16.803275 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.803255 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24d94c22-6600-483e-9562-549707afa24d" containerName="util" Sep 09 23:22:16.803275 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.803261 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="24d94c22-6600-483e-9562-549707afa24d" containerName="util" Sep 09 23:22:16.803426 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.803311 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="24d94c22-6600-483e-9562-549707afa24d" containerName="extract" Sep 09 23:22:16.805686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.805670 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x" Sep 09 23:22:16.808010 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.807982 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 09 23:22:16.808185 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.808164 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-8p4w5\"" Sep 09 23:22:16.808276 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.808238 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 09 23:22:16.818799 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.818773 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x"] Sep 09 23:22:16.877646 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.877611 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6025c052-7622-47b5-aaf7-902c23f42bd3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rlw7x\" (UID: \"6025c052-7622-47b5-aaf7-902c23f42bd3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x" Sep 09 23:22:16.877810 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.877657 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqcvd\" (UniqueName: \"kubernetes.io/projected/6025c052-7622-47b5-aaf7-902c23f42bd3-kube-api-access-mqcvd\") pod \"openshift-lws-operator-fd8ccff4c-rlw7x\" (UID: \"6025c052-7622-47b5-aaf7-902c23f42bd3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x" Sep 09 23:22:16.978055 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.978015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6025c052-7622-47b5-aaf7-902c23f42bd3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rlw7x\" (UID: \"6025c052-7622-47b5-aaf7-902c23f42bd3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x" Sep 09 23:22:16.978241 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.978065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqcvd\" (UniqueName: \"kubernetes.io/projected/6025c052-7622-47b5-aaf7-902c23f42bd3-kube-api-access-mqcvd\") pod \"openshift-lws-operator-fd8ccff4c-rlw7x\" (UID: \"6025c052-7622-47b5-aaf7-902c23f42bd3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x" Sep 09 23:22:16.978405 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.978385 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6025c052-7622-47b5-aaf7-902c23f42bd3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rlw7x\" (UID: \"6025c052-7622-47b5-aaf7-902c23f42bd3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x" Sep 09 23:22:16.990760 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:16.990736 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqcvd\" (UniqueName: \"kubernetes.io/projected/6025c052-7622-47b5-aaf7-902c23f42bd3-kube-api-access-mqcvd\") pod \"openshift-lws-operator-fd8ccff4c-rlw7x\" (UID: \"6025c052-7622-47b5-aaf7-902c23f42bd3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x" Sep 09 23:22:17.114805 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:17.114721 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x" Sep 09 23:22:17.382613 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:17.382584 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x"] Sep 09 23:22:17.385065 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:22:17.385037 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6025c052_7622_47b5_aaf7_902c23f42bd3.slice/crio-1c456faaad474a795fca10c3ac1c79c9ff122c01e4ac31598936ec5a88c34052 WatchSource:0}: Error finding container 1c456faaad474a795fca10c3ac1c79c9ff122c01e4ac31598936ec5a88c34052: Status 404 returned error can't find the container with id 1c456faaad474a795fca10c3ac1c79c9ff122c01e4ac31598936ec5a88c34052 Sep 09 23:22:17.408419 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:17.408389 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x" event={"ID":"6025c052-7622-47b5-aaf7-902c23f42bd3","Type":"ContainerStarted","Data":"1c456faaad474a795fca10c3ac1c79c9ff122c01e4ac31598936ec5a88c34052"} Sep 09 23:22:19.097265 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:19.097236 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/0.log" Sep 09 23:22:19.097740 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:19.097557 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/0.log" Sep 09 23:22:19.101094 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:19.101072 2580 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 23:22:19.420695 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:19.420659 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x" event={"ID":"6025c052-7622-47b5-aaf7-902c23f42bd3","Type":"ContainerStarted","Data":"ea1012c38273b2e07c360bb510f4ad7cad341174fa5f7bd6f25c7762d3a65e98"} Sep 09 23:22:19.436433 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:19.436386 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rlw7x" podStartSLOduration=1.716668345 podStartE2EDuration="3.436370903s" totalImagesPullingTime="1.719702558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:22:19.435315046 +0000 UTC m=+300.861738357" watchObservedRunningTime="2026-09-09 23:22:19.436370903 +0000 UTC m=+300.862794210" Sep 09 23:22:22.018758 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.018718 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z"] Sep 09 23:22:22.021024 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.021006 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:22.023427 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.023406 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 09 23:22:22.023541 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.023472 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 09 23:22:22.023636 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.023616 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bkm2k\"" Sep 09 23:22:22.029305 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.029281 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z"] Sep 09 23:22:22.119132 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.119097 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36948c46-8ab9-411c-a3b6-a8be60df1ec3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z\" (UID: \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:22.119297 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.119166 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjqnp\" (UniqueName: \"kubernetes.io/projected/36948c46-8ab9-411c-a3b6-a8be60df1ec3-kube-api-access-xjqnp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z\" (UID: \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:22.119297 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.119215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36948c46-8ab9-411c-a3b6-a8be60df1ec3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z\" (UID: \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:22.219940 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.219908 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36948c46-8ab9-411c-a3b6-a8be60df1ec3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z\" (UID: \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:22.220112 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.219957 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36948c46-8ab9-411c-a3b6-a8be60df1ec3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z\" (UID: \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:22.220112 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.220040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjqnp\" (UniqueName: \"kubernetes.io/projected/36948c46-8ab9-411c-a3b6-a8be60df1ec3-kube-api-access-xjqnp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z\" (UID: \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:22.220337 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.220317 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36948c46-8ab9-411c-a3b6-a8be60df1ec3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z\" (UID: \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:22.220385 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.220363 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36948c46-8ab9-411c-a3b6-a8be60df1ec3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z\" (UID: \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:22.227184 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.227163 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjqnp\" (UniqueName: \"kubernetes.io/projected/36948c46-8ab9-411c-a3b6-a8be60df1ec3-kube-api-access-xjqnp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z\" (UID: \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:22.330012 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.329980 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:22.598234 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:22.598185 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z"] Sep 09 23:22:22.600877 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:22:22.600847 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36948c46_8ab9_411c_a3b6_a8be60df1ec3.slice/crio-1bddc9151f6540bb395a5c6f1c2574e6b0043192fd83ed56cc02f04086950d1a WatchSource:0}: Error finding container 1bddc9151f6540bb395a5c6f1c2574e6b0043192fd83ed56cc02f04086950d1a: Status 404 returned error can't find the container with id 1bddc9151f6540bb395a5c6f1c2574e6b0043192fd83ed56cc02f04086950d1a Sep 09 23:22:23.440418 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:23.440382 2580 generic.go:356] "Generic (PLEG): container finished" podID="36948c46-8ab9-411c-a3b6-a8be60df1ec3" containerID="edec8fcd7a3ad845bf8dc2eaaa4ea6c3c04ed982ed5309f7cafb7c8367a1c69b" exitCode=0 Sep 09 23:22:23.440845 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:23.440433 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" event={"ID":"36948c46-8ab9-411c-a3b6-a8be60df1ec3","Type":"ContainerDied","Data":"edec8fcd7a3ad845bf8dc2eaaa4ea6c3c04ed982ed5309f7cafb7c8367a1c69b"} Sep 09 23:22:23.440845 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:23.440467 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" event={"ID":"36948c46-8ab9-411c-a3b6-a8be60df1ec3","Type":"ContainerStarted","Data":"1bddc9151f6540bb395a5c6f1c2574e6b0043192fd83ed56cc02f04086950d1a"} Sep 09 23:22:23.440923 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:23.440877 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 23:22:24.446169 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:24.446125 2580 generic.go:356] "Generic (PLEG): container finished" podID="36948c46-8ab9-411c-a3b6-a8be60df1ec3" containerID="741e6dc4629fe4e371f65bc26dae14df90be865d1c30ce3887787afe5456c873" exitCode=0 Sep 09 23:22:24.446546 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:24.446192 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" event={"ID":"36948c46-8ab9-411c-a3b6-a8be60df1ec3","Type":"ContainerDied","Data":"741e6dc4629fe4e371f65bc26dae14df90be865d1c30ce3887787afe5456c873"} Sep 09 23:22:25.453448 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:25.453416 2580 generic.go:356] "Generic (PLEG): container finished" podID="36948c46-8ab9-411c-a3b6-a8be60df1ec3" containerID="d9d00a20b4bf4c824fc7ff17143619c06a965a8cc5191e0b2d057cfbdb575124" exitCode=0 Sep 09 23:22:25.453866 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:25.453475 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" event={"ID":"36948c46-8ab9-411c-a3b6-a8be60df1ec3","Type":"ContainerDied","Data":"d9d00a20b4bf4c824fc7ff17143619c06a965a8cc5191e0b2d057cfbdb575124"} Sep 09 23:22:26.572005 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:26.571980 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:26.661899 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:26.661861 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36948c46-8ab9-411c-a3b6-a8be60df1ec3-bundle\") pod \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\" (UID: \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\") " Sep 09 23:22:26.662083 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:26.662006 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjqnp\" (UniqueName: \"kubernetes.io/projected/36948c46-8ab9-411c-a3b6-a8be60df1ec3-kube-api-access-xjqnp\") pod \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\" (UID: \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\") " Sep 09 23:22:26.662083 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:26.662061 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36948c46-8ab9-411c-a3b6-a8be60df1ec3-util\") pod \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\" (UID: \"36948c46-8ab9-411c-a3b6-a8be60df1ec3\") " Sep 09 23:22:26.662714 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:26.662687 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36948c46-8ab9-411c-a3b6-a8be60df1ec3-bundle" (OuterVolumeSpecName: "bundle") pod "36948c46-8ab9-411c-a3b6-a8be60df1ec3" (UID: "36948c46-8ab9-411c-a3b6-a8be60df1ec3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:22:26.664057 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:26.664030 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36948c46-8ab9-411c-a3b6-a8be60df1ec3-kube-api-access-xjqnp" (OuterVolumeSpecName: "kube-api-access-xjqnp") pod "36948c46-8ab9-411c-a3b6-a8be60df1ec3" (UID: "36948c46-8ab9-411c-a3b6-a8be60df1ec3"). InnerVolumeSpecName "kube-api-access-xjqnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:22:26.667691 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:26.667661 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36948c46-8ab9-411c-a3b6-a8be60df1ec3-util" (OuterVolumeSpecName: "util") pod "36948c46-8ab9-411c-a3b6-a8be60df1ec3" (UID: "36948c46-8ab9-411c-a3b6-a8be60df1ec3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:22:26.763650 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:26.763564 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjqnp\" (UniqueName: \"kubernetes.io/projected/36948c46-8ab9-411c-a3b6-a8be60df1ec3-kube-api-access-xjqnp\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:22:26.763650 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:26.763597 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36948c46-8ab9-411c-a3b6-a8be60df1ec3-util\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:22:26.763650 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:26.763609 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36948c46-8ab9-411c-a3b6-a8be60df1ec3-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:22:27.463494 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:27.463410 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" event={"ID":"36948c46-8ab9-411c-a3b6-a8be60df1ec3","Type":"ContainerDied","Data":"1bddc9151f6540bb395a5c6f1c2574e6b0043192fd83ed56cc02f04086950d1a"} Sep 09 23:22:27.463494 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:27.463429 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9w54z" Sep 09 23:22:27.463494 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:27.463460 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1bddc9151f6540bb395a5c6f1c2574e6b0043192fd83ed56cc02f04086950d1a" Sep 09 23:22:38.630691 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.630652 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9"] Sep 09 23:22:38.631097 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.630950 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36948c46-8ab9-411c-a3b6-a8be60df1ec3" containerName="pull" Sep 09 23:22:38.631097 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.630961 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="36948c46-8ab9-411c-a3b6-a8be60df1ec3" containerName="pull" Sep 09 23:22:38.631097 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.630990 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36948c46-8ab9-411c-a3b6-a8be60df1ec3" containerName="util" Sep 09 23:22:38.631097 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.630998 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="36948c46-8ab9-411c-a3b6-a8be60df1ec3" containerName="util" Sep 09 23:22:38.631097 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.631020 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36948c46-8ab9-411c-a3b6-a8be60df1ec3" containerName="extract" Sep 09 23:22:38.631097 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.631029 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="36948c46-8ab9-411c-a3b6-a8be60df1ec3" containerName="extract" Sep 09 23:22:38.631097 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.631081 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="36948c46-8ab9-411c-a3b6-a8be60df1ec3" containerName="extract" Sep 09 23:22:38.635163 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.635146 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:38.637680 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.637640 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 09 23:22:38.638546 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.638528 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 09 23:22:38.638646 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.638535 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bkm2k\"" Sep 09 23:22:38.641519 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.641490 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9"] Sep 09 23:22:38.765537 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.765504 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47rnw\" (UniqueName: \"kubernetes.io/projected/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-kube-api-access-47rnw\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9\" (UID: \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:38.765735 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.765629 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9\" (UID: \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:38.765735 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.765654 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9\" (UID: \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:38.866095 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.866065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9\" (UID: \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:38.866095 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.866096 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9\" (UID: \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:38.866303 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.866126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47rnw\" (UniqueName: \"kubernetes.io/projected/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-kube-api-access-47rnw\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9\" (UID: \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:38.866565 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.866531 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9\" (UID: \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:38.866598 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.866542 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9\" (UID: \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:38.879551 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.879521 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47rnw\" (UniqueName: \"kubernetes.io/projected/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-kube-api-access-47rnw\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9\" (UID: \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:38.945108 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:38.945035 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:39.209993 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.209955 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9"] Sep 09 23:22:39.212069 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:22:39.212038 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a70062_18cc_40e2_b7ac_50f921c4a9d6.slice/crio-8e628535474d293058eb106bb857f02441140470422e431bf58e7fe67251d54b WatchSource:0}: Error finding container 8e628535474d293058eb106bb857f02441140470422e431bf58e7fe67251d54b: Status 404 returned error can't find the container with id 8e628535474d293058eb106bb857f02441140470422e431bf58e7fe67251d54b Sep 09 23:22:39.516412 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.516329 2580 generic.go:356] "Generic (PLEG): container finished" podID="b6a70062-18cc-40e2-b7ac-50f921c4a9d6" containerID="effdeb8633bcb7117ae247abcf2e9672504928e60ca58333330d9e371a371e0e" exitCode=0 Sep 09 23:22:39.516412 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.516369 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" event={"ID":"b6a70062-18cc-40e2-b7ac-50f921c4a9d6","Type":"ContainerDied","Data":"effdeb8633bcb7117ae247abcf2e9672504928e60ca58333330d9e371a371e0e"} Sep 09 23:22:39.516412 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.516406 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" event={"ID":"b6a70062-18cc-40e2-b7ac-50f921c4a9d6","Type":"ContainerStarted","Data":"8e628535474d293058eb106bb857f02441140470422e431bf58e7fe67251d54b"} Sep 09 23:22:39.760389 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.760355 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh"] Sep 09 23:22:39.762578 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.762555 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.765185 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.765155 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 09 23:22:39.765580 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.765538 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 09 23:22:39.765739 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.765717 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 09 23:22:39.765873 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.765856 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 09 23:22:39.765939 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.765920 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7xcjn\"" Sep 09 23:22:39.786267 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.786239 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh"] Sep 09 23:22:39.874622 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.874594 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lqjg\" (UniqueName: \"kubernetes.io/projected/854fe1c7-968f-457c-b31d-caaa41a22153-kube-api-access-9lqjg\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.874759 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.874632 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/854fe1c7-968f-457c-b31d-caaa41a22153-webhook-cert\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.874759 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.874655 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/854fe1c7-968f-457c-b31d-caaa41a22153-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.874881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.874759 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/854fe1c7-968f-457c-b31d-caaa41a22153-tmp\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.874881 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.874810 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/854fe1c7-968f-457c-b31d-caaa41a22153-manifests\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.976053 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.976019 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/854fe1c7-968f-457c-b31d-caaa41a22153-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.976218 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.976075 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/854fe1c7-968f-457c-b31d-caaa41a22153-tmp\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.976218 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.976107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/854fe1c7-968f-457c-b31d-caaa41a22153-manifests\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.976218 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.976137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lqjg\" (UniqueName: \"kubernetes.io/projected/854fe1c7-968f-457c-b31d-caaa41a22153-kube-api-access-9lqjg\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.976218 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.976166 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/854fe1c7-968f-457c-b31d-caaa41a22153-webhook-cert\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.976892 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.976863 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/854fe1c7-968f-457c-b31d-caaa41a22153-manifests\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.978514 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.978461 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/854fe1c7-968f-457c-b31d-caaa41a22153-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.978627 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.978488 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/854fe1c7-968f-457c-b31d-caaa41a22153-webhook-cert\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.978627 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.978552 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/854fe1c7-968f-457c-b31d-caaa41a22153-tmp\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:39.984011 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:39.983953 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lqjg\" (UniqueName: \"kubernetes.io/projected/854fe1c7-968f-457c-b31d-caaa41a22153-kube-api-access-9lqjg\") pod \"opendatahub-operator-controller-manager-8c5c8967d-rwzxh\" (UID: \"854fe1c7-968f-457c-b31d-caaa41a22153\") " pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:40.073513 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:40.073480 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:40.351935 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:40.351911 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh"] Sep 09 23:22:40.353980 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:22:40.353936 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod854fe1c7_968f_457c_b31d_caaa41a22153.slice/crio-84a9ee0f0391dcec432d3fcccfe3dc8ecb05d60c78f8a74b7b499fc55f76424b WatchSource:0}: Error finding container 84a9ee0f0391dcec432d3fcccfe3dc8ecb05d60c78f8a74b7b499fc55f76424b: Status 404 returned error can't find the container with id 84a9ee0f0391dcec432d3fcccfe3dc8ecb05d60c78f8a74b7b499fc55f76424b Sep 09 23:22:40.522161 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:40.522118 2580 generic.go:356] "Generic (PLEG): container finished" podID="b6a70062-18cc-40e2-b7ac-50f921c4a9d6" containerID="3f3de60556efe14d580d0615f18e6bfc73c5c4df8d9e05fe89e48215beff7e47" exitCode=0 Sep 09 23:22:40.522353 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:40.522203 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" event={"ID":"b6a70062-18cc-40e2-b7ac-50f921c4a9d6","Type":"ContainerDied","Data":"3f3de60556efe14d580d0615f18e6bfc73c5c4df8d9e05fe89e48215beff7e47"} Sep 09 23:22:40.523606 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:40.523578 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" event={"ID":"854fe1c7-968f-457c-b31d-caaa41a22153","Type":"ContainerStarted","Data":"84a9ee0f0391dcec432d3fcccfe3dc8ecb05d60c78f8a74b7b499fc55f76424b"} Sep 09 23:22:41.532504 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:41.532467 2580 generic.go:356] "Generic (PLEG): container finished" podID="b6a70062-18cc-40e2-b7ac-50f921c4a9d6" containerID="9221bd4e96377f3445a0a24185278b125687dc10403bb31baa2803b035d2bf45" exitCode=0 Sep 09 23:22:41.533066 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:41.532603 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" event={"ID":"b6a70062-18cc-40e2-b7ac-50f921c4a9d6","Type":"ContainerDied","Data":"9221bd4e96377f3445a0a24185278b125687dc10403bb31baa2803b035d2bf45"} Sep 09 23:22:43.191308 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.191282 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:43.197262 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.197240 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-bundle\") pod \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\" (UID: \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\") " Sep 09 23:22:43.197367 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.197319 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-util\") pod \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\" (UID: \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\") " Sep 09 23:22:43.197367 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.197338 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47rnw\" (UniqueName: \"kubernetes.io/projected/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-kube-api-access-47rnw\") pod \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\" (UID: \"b6a70062-18cc-40e2-b7ac-50f921c4a9d6\") " Sep 09 23:22:43.197938 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.197910 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-bundle" (OuterVolumeSpecName: "bundle") pod "b6a70062-18cc-40e2-b7ac-50f921c4a9d6" (UID: "b6a70062-18cc-40e2-b7ac-50f921c4a9d6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:22:43.199389 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.199359 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-kube-api-access-47rnw" (OuterVolumeSpecName: "kube-api-access-47rnw") pod "b6a70062-18cc-40e2-b7ac-50f921c4a9d6" (UID: "b6a70062-18cc-40e2-b7ac-50f921c4a9d6"). InnerVolumeSpecName "kube-api-access-47rnw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:22:43.202422 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.202397 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-util" (OuterVolumeSpecName: "util") pod "b6a70062-18cc-40e2-b7ac-50f921c4a9d6" (UID: "b6a70062-18cc-40e2-b7ac-50f921c4a9d6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:22:43.297737 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.297708 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-util\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:22:43.297737 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.297733 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47rnw\" (UniqueName: \"kubernetes.io/projected/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-kube-api-access-47rnw\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:22:43.297869 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.297743 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6a70062-18cc-40e2-b7ac-50f921c4a9d6-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:22:43.544481 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.544455 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" Sep 09 23:22:43.544640 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.544455 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0cwwx9" event={"ID":"b6a70062-18cc-40e2-b7ac-50f921c4a9d6","Type":"ContainerDied","Data":"8e628535474d293058eb106bb857f02441140470422e431bf58e7fe67251d54b"} Sep 09 23:22:43.544640 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.544610 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e628535474d293058eb106bb857f02441140470422e431bf58e7fe67251d54b" Sep 09 23:22:43.546137 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:43.546114 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" event={"ID":"854fe1c7-968f-457c-b31d-caaa41a22153","Type":"ContainerStarted","Data":"50c7bc42fb5ad627200851a1f72cf65867fd92073f88e30639d729250f011fea"} Sep 09 23:22:44.552247 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:44.552210 2580 generic.go:356] "Generic (PLEG): container finished" podID="854fe1c7-968f-457c-b31d-caaa41a22153" containerID="50c7bc42fb5ad627200851a1f72cf65867fd92073f88e30639d729250f011fea" exitCode=0 Sep 09 23:22:44.552612 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:44.552296 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" event={"ID":"854fe1c7-968f-457c-b31d-caaa41a22153","Type":"ContainerDied","Data":"50c7bc42fb5ad627200851a1f72cf65867fd92073f88e30639d729250f011fea"} Sep 09 23:22:45.558357 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:45.558322 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" event={"ID":"854fe1c7-968f-457c-b31d-caaa41a22153","Type":"ContainerStarted","Data":"e7e2d649f64841920da8c5b0ac03d9668ddcce7c49b2b608e19f9c71f9a5f498"} Sep 09 23:22:45.558740 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:45.558423 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:56.564476 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:56.564445 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" Sep 09 23:22:56.589397 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:22:56.589351 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8c5c8967d-rwzxh" podStartSLOduration=13.205792538 podStartE2EDuration="17.58933725s" totalImagesPullingTime="3.305420683s" totalInitContainerRuntime="1.078124029s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:22:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:22:45.604164786 +0000 UTC m=+327.030588094" watchObservedRunningTime="2026-09-09 23:22:56.58933725 +0000 UTC m=+338.015760558" Sep 09 23:23:00.131081 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.131044 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8"] Sep 09 23:23:00.131428 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.131310 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6a70062-18cc-40e2-b7ac-50f921c4a9d6" containerName="pull" Sep 09 23:23:00.131428 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.131321 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6a70062-18cc-40e2-b7ac-50f921c4a9d6" containerName="pull" Sep 09 23:23:00.131428 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.131329 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6a70062-18cc-40e2-b7ac-50f921c4a9d6" containerName="extract" Sep 09 23:23:00.131428 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.131334 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6a70062-18cc-40e2-b7ac-50f921c4a9d6" containerName="extract" Sep 09 23:23:00.131428 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.131351 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6a70062-18cc-40e2-b7ac-50f921c4a9d6" containerName="util" Sep 09 23:23:00.131428 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.131356 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6a70062-18cc-40e2-b7ac-50f921c4a9d6" containerName="util" Sep 09 23:23:00.131428 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.131405 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6a70062-18cc-40e2-b7ac-50f921c4a9d6" containerName="extract" Sep 09 23:23:00.139942 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.139919 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" Sep 09 23:23:00.143052 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.143032 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 09 23:23:00.143196 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.143176 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 09 23:23:00.143706 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.143690 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 09 23:23:00.144257 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.144241 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 09 23:23:00.155430 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.155409 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8"] Sep 09 23:23:00.217188 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.217161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a250e59-3323-4852-9e47-4cc738212fa0-tmp\") pod \"kube-auth-proxy-7bd8cf678b-bj5d8\" (UID: \"6a250e59-3323-4852-9e47-4cc738212fa0\") " pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" Sep 09 23:23:00.217598 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.217572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7tmx\" (UniqueName: \"kubernetes.io/projected/6a250e59-3323-4852-9e47-4cc738212fa0-kube-api-access-l7tmx\") pod \"kube-auth-proxy-7bd8cf678b-bj5d8\" (UID: \"6a250e59-3323-4852-9e47-4cc738212fa0\") " pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" Sep 09 23:23:00.217763 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.217737 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a250e59-3323-4852-9e47-4cc738212fa0-tls-certs\") pod \"kube-auth-proxy-7bd8cf678b-bj5d8\" (UID: \"6a250e59-3323-4852-9e47-4cc738212fa0\") " pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" Sep 09 23:23:00.319142 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.319107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a250e59-3323-4852-9e47-4cc738212fa0-tmp\") pod \"kube-auth-proxy-7bd8cf678b-bj5d8\" (UID: \"6a250e59-3323-4852-9e47-4cc738212fa0\") " pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" Sep 09 23:23:00.319298 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.319174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7tmx\" (UniqueName: \"kubernetes.io/projected/6a250e59-3323-4852-9e47-4cc738212fa0-kube-api-access-l7tmx\") pod \"kube-auth-proxy-7bd8cf678b-bj5d8\" (UID: \"6a250e59-3323-4852-9e47-4cc738212fa0\") " pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" Sep 09 23:23:00.319298 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.319203 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a250e59-3323-4852-9e47-4cc738212fa0-tls-certs\") pod \"kube-auth-proxy-7bd8cf678b-bj5d8\" (UID: \"6a250e59-3323-4852-9e47-4cc738212fa0\") " pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" Sep 09 23:23:00.321364 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.321337 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a250e59-3323-4852-9e47-4cc738212fa0-tmp\") pod \"kube-auth-proxy-7bd8cf678b-bj5d8\" (UID: \"6a250e59-3323-4852-9e47-4cc738212fa0\") " pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" Sep 09 23:23:00.321582 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.321560 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a250e59-3323-4852-9e47-4cc738212fa0-tls-certs\") pod \"kube-auth-proxy-7bd8cf678b-bj5d8\" (UID: \"6a250e59-3323-4852-9e47-4cc738212fa0\") " pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" Sep 09 23:23:00.328232 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.328212 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7tmx\" (UniqueName: \"kubernetes.io/projected/6a250e59-3323-4852-9e47-4cc738212fa0-kube-api-access-l7tmx\") pod \"kube-auth-proxy-7bd8cf678b-bj5d8\" (UID: \"6a250e59-3323-4852-9e47-4cc738212fa0\") " pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" Sep 09 23:23:00.448773 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.448695 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" Sep 09 23:23:00.719639 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:00.719613 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8"] Sep 09 23:23:00.721875 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:23:00.721844 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a250e59_3323_4852_9e47_4cc738212fa0.slice/crio-cfd281075ee878af42019f39f05637a975d2da4611b90499d6b9bfb9553a2695 WatchSource:0}: Error finding container cfd281075ee878af42019f39f05637a975d2da4611b90499d6b9bfb9553a2695: Status 404 returned error can't find the container with id cfd281075ee878af42019f39f05637a975d2da4611b90499d6b9bfb9553a2695 Sep 09 23:23:01.633641 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:01.633592 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" event={"ID":"6a250e59-3323-4852-9e47-4cc738212fa0","Type":"ContainerStarted","Data":"cfd281075ee878af42019f39f05637a975d2da4611b90499d6b9bfb9553a2695"} Sep 09 23:23:02.980654 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:02.980624 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f46cd58f7-ds2nf"] Sep 09 23:23:02.984329 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:02.984306 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:02.993892 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:02.993863 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f46cd58f7-ds2nf"] Sep 09 23:23:03.044676 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.044638 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/546492f4-9265-41a1-9f87-9e07df18e82e-oauth-serving-cert\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.044855 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.044683 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/546492f4-9265-41a1-9f87-9e07df18e82e-console-serving-cert\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.044855 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.044751 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/546492f4-9265-41a1-9f87-9e07df18e82e-console-oauth-config\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.044855 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.044848 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/546492f4-9265-41a1-9f87-9e07df18e82e-service-ca\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.045052 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.044876 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfjj5\" (UniqueName: \"kubernetes.io/projected/546492f4-9265-41a1-9f87-9e07df18e82e-kube-api-access-gfjj5\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.045052 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.044909 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/546492f4-9265-41a1-9f87-9e07df18e82e-trusted-ca-bundle\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.045052 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.044984 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/546492f4-9265-41a1-9f87-9e07df18e82e-console-config\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.145550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.145515 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/546492f4-9265-41a1-9f87-9e07df18e82e-console-oauth-config\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.145714 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.145589 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/546492f4-9265-41a1-9f87-9e07df18e82e-service-ca\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.145714 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.145618 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfjj5\" (UniqueName: \"kubernetes.io/projected/546492f4-9265-41a1-9f87-9e07df18e82e-kube-api-access-gfjj5\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.145714 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.145648 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/546492f4-9265-41a1-9f87-9e07df18e82e-trusted-ca-bundle\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.145714 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.145685 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/546492f4-9265-41a1-9f87-9e07df18e82e-console-config\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.145940 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.145735 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/546492f4-9265-41a1-9f87-9e07df18e82e-oauth-serving-cert\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.145940 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.145765 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/546492f4-9265-41a1-9f87-9e07df18e82e-console-serving-cert\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.146684 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.146653 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/546492f4-9265-41a1-9f87-9e07df18e82e-service-ca\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.146806 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.146788 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/546492f4-9265-41a1-9f87-9e07df18e82e-trusted-ca-bundle\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.147207 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.147184 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/546492f4-9265-41a1-9f87-9e07df18e82e-console-config\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.147278 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.147187 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/546492f4-9265-41a1-9f87-9e07df18e82e-oauth-serving-cert\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.148304 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.148281 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/546492f4-9265-41a1-9f87-9e07df18e82e-console-oauth-config\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.148575 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.148553 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/546492f4-9265-41a1-9f87-9e07df18e82e-console-serving-cert\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.159086 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.159057 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfjj5\" (UniqueName: \"kubernetes.io/projected/546492f4-9265-41a1-9f87-9e07df18e82e-kube-api-access-gfjj5\") pod \"console-f46cd58f7-ds2nf\" (UID: \"546492f4-9265-41a1-9f87-9e07df18e82e\") " pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.296539 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.296501 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:03.622441 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.622414 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f46cd58f7-ds2nf"] Sep 09 23:23:03.624558 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:23:03.624531 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod546492f4_9265_41a1_9f87_9e07df18e82e.slice/crio-d3ce6d06b40244ac40e3158297cd00059241322c03e7341f980e6b1c49b910db WatchSource:0}: Error finding container d3ce6d06b40244ac40e3158297cd00059241322c03e7341f980e6b1c49b910db: Status 404 returned error can't find the container with id d3ce6d06b40244ac40e3158297cd00059241322c03e7341f980e6b1c49b910db Sep 09 23:23:03.643775 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.643740 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" event={"ID":"6a250e59-3323-4852-9e47-4cc738212fa0","Type":"ContainerStarted","Data":"aee6d5346904154e4b56e941170350235aa7b34f13429776e6add81d5ec37a57"} Sep 09 23:23:03.645041 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.645015 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f46cd58f7-ds2nf" event={"ID":"546492f4-9265-41a1-9f87-9e07df18e82e","Type":"ContainerStarted","Data":"d3ce6d06b40244ac40e3158297cd00059241322c03e7341f980e6b1c49b910db"} Sep 09 23:23:03.659503 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:03.659455 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7bd8cf678b-bj5d8" podStartSLOduration=0.966011662 podStartE2EDuration="3.659437591s" totalImagesPullingTime="2.693425929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:23:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:23:03.658574857 +0000 UTC m=+345.084998168" watchObservedRunningTime="2026-09-09 23:23:03.659437591 +0000 UTC m=+345.085860920" Sep 09 23:23:04.650722 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:04.650676 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f46cd58f7-ds2nf" event={"ID":"546492f4-9265-41a1-9f87-9e07df18e82e","Type":"ContainerStarted","Data":"c7e140f62b79bc5906716eb57d0f3a1851a902d48637a6dbdc39681d3078eafc"} Sep 09 23:23:04.668229 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:04.668185 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f46cd58f7-ds2nf" podStartSLOduration=2.6681702229999997 podStartE2EDuration="2.668170223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:23:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:23:04.666763412 +0000 UTC m=+346.093186720" watchObservedRunningTime="2026-09-09 23:23:04.668170223 +0000 UTC m=+346.094593530" Sep 09 23:23:06.679335 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.679300 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5"] Sep 09 23:23:06.682505 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.682479 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.685029 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.685006 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 09 23:23:06.685158 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.685005 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-mjckr\"" Sep 09 23:23:06.693751 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.693727 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5"] Sep 09 23:23:06.778268 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.778239 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.778441 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.778275 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35ec7208-3462-40cc-8a7b-816a5d860510-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.778441 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.778306 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbnpg\" (UniqueName: \"kubernetes.io/projected/35ec7208-3462-40cc-8a7b-816a5d860510-kube-api-access-dbnpg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.778441 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.778418 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35ec7208-3462-40cc-8a7b-816a5d860510-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.778551 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.778457 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.778551 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.778472 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.778551 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.778530 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.778651 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.778562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35ec7208-3462-40cc-8a7b-816a5d860510-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.778651 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.778598 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879197 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35ec7208-3462-40cc-8a7b-816a5d860510-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879197 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbnpg\" (UniqueName: \"kubernetes.io/projected/35ec7208-3462-40cc-8a7b-816a5d860510-kube-api-access-dbnpg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879429 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35ec7208-3462-40cc-8a7b-816a5d860510-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879429 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879252 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879429 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879429 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879429 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35ec7208-3462-40cc-8a7b-816a5d860510-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879429 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879429 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879402 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879810 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879761 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879871 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879813 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879871 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879833 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.879944 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.879925 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.880271 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.880246 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35ec7208-3462-40cc-8a7b-816a5d860510-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.881553 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.881532 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35ec7208-3462-40cc-8a7b-816a5d860510-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.881715 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.881700 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35ec7208-3462-40cc-8a7b-816a5d860510-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.888502 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.888477 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35ec7208-3462-40cc-8a7b-816a5d860510-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.888602 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.888584 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbnpg\" (UniqueName: \"kubernetes.io/projected/35ec7208-3462-40cc-8a7b-816a5d860510-kube-api-access-dbnpg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5\" (UID: \"35ec7208-3462-40cc-8a7b-816a5d860510\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:06.992522 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:06.992496 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:07.267368 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:07.267344 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5"] Sep 09 23:23:07.269114 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:23:07.269087 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35ec7208_3462_40cc_8a7b_816a5d860510.slice/crio-4a4762f0d7da5099f6feace3fd3b85c2b69f5ddedcfe4e989f0b02559c739c54 WatchSource:0}: Error finding container 4a4762f0d7da5099f6feace3fd3b85c2b69f5ddedcfe4e989f0b02559c739c54: Status 404 returned error can't find the container with id 4a4762f0d7da5099f6feace3fd3b85c2b69f5ddedcfe4e989f0b02559c739c54 Sep 09 23:23:07.666772 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:07.666735 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" event={"ID":"35ec7208-3462-40cc-8a7b-816a5d860510","Type":"ContainerStarted","Data":"4a4762f0d7da5099f6feace3fd3b85c2b69f5ddedcfe4e989f0b02559c739c54"} Sep 09 23:23:09.450149 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:09.450111 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nmx4g"] Sep 09 23:23:09.457121 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:09.457096 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" Sep 09 23:23:09.460632 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:09.460600 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 09 23:23:09.460843 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:09.460825 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6n7nt\"" Sep 09 23:23:09.465726 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:09.465662 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nmx4g"] Sep 09 23:23:09.504450 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:09.504413 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2psb9\" (UniqueName: \"kubernetes.io/projected/f594896f-45b1-477d-8e1a-419edd1bc2c6-kube-api-access-2psb9\") pod \"odh-model-controller-7998c67dc4-nmx4g\" (UID: \"f594896f-45b1-477d-8e1a-419edd1bc2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" Sep 09 23:23:09.504621 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:09.504544 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f594896f-45b1-477d-8e1a-419edd1bc2c6-cert\") pod \"odh-model-controller-7998c67dc4-nmx4g\" (UID: \"f594896f-45b1-477d-8e1a-419edd1bc2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" Sep 09 23:23:09.605433 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:09.605383 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2psb9\" (UniqueName: \"kubernetes.io/projected/f594896f-45b1-477d-8e1a-419edd1bc2c6-kube-api-access-2psb9\") pod \"odh-model-controller-7998c67dc4-nmx4g\" (UID: \"f594896f-45b1-477d-8e1a-419edd1bc2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" Sep 09 23:23:09.605614 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:09.605507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f594896f-45b1-477d-8e1a-419edd1bc2c6-cert\") pod \"odh-model-controller-7998c67dc4-nmx4g\" (UID: \"f594896f-45b1-477d-8e1a-419edd1bc2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" Sep 09 23:23:09.605676 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:23:09.605627 2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 09 23:23:09.605728 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:23:09.605689 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f594896f-45b1-477d-8e1a-419edd1bc2c6-cert podName:f594896f-45b1-477d-8e1a-419edd1bc2c6 nodeName:}" failed. No retries permitted until 2026-09-09 23:23:10.10566704 +0000 UTC m=+351.532090329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f594896f-45b1-477d-8e1a-419edd1bc2c6-cert") pod "odh-model-controller-7998c67dc4-nmx4g" (UID: "f594896f-45b1-477d-8e1a-419edd1bc2c6") : secret "odh-model-controller-webhook-cert" not found Sep 09 23:23:09.613574 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:09.613548 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2psb9\" (UniqueName: \"kubernetes.io/projected/f594896f-45b1-477d-8e1a-419edd1bc2c6-kube-api-access-2psb9\") pod \"odh-model-controller-7998c67dc4-nmx4g\" (UID: \"f594896f-45b1-477d-8e1a-419edd1bc2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" Sep 09 23:23:10.003348 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.003317 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9nx9s"] Sep 09 23:23:10.006681 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.006660 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" Sep 09 23:23:10.008987 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.008954 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 09 23:23:10.009106 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.009025 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-wc279\"" Sep 09 23:23:10.016815 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.016790 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9nx9s"] Sep 09 23:23:10.110231 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.110199 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e42903f1-61a7-44b9-8062-8ee4e1b138b0-tls-certs\") pod \"model-serving-api-54d89bd5fc-9nx9s\" (UID: \"e42903f1-61a7-44b9-8062-8ee4e1b138b0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" Sep 09 23:23:10.110393 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.110256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfqtw\" (UniqueName: \"kubernetes.io/projected/e42903f1-61a7-44b9-8062-8ee4e1b138b0-kube-api-access-cfqtw\") pod \"model-serving-api-54d89bd5fc-9nx9s\" (UID: \"e42903f1-61a7-44b9-8062-8ee4e1b138b0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" Sep 09 23:23:10.110393 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.110358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f594896f-45b1-477d-8e1a-419edd1bc2c6-cert\") pod \"odh-model-controller-7998c67dc4-nmx4g\" (UID: \"f594896f-45b1-477d-8e1a-419edd1bc2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" Sep 09 23:23:10.112767 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.112744 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f594896f-45b1-477d-8e1a-419edd1bc2c6-cert\") pod \"odh-model-controller-7998c67dc4-nmx4g\" (UID: \"f594896f-45b1-477d-8e1a-419edd1bc2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" Sep 09 23:23:10.211056 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.211022 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfqtw\" (UniqueName: \"kubernetes.io/projected/e42903f1-61a7-44b9-8062-8ee4e1b138b0-kube-api-access-cfqtw\") pod \"model-serving-api-54d89bd5fc-9nx9s\" (UID: \"e42903f1-61a7-44b9-8062-8ee4e1b138b0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" Sep 09 23:23:10.211280 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.211136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e42903f1-61a7-44b9-8062-8ee4e1b138b0-tls-certs\") pod \"model-serving-api-54d89bd5fc-9nx9s\" (UID: \"e42903f1-61a7-44b9-8062-8ee4e1b138b0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" Sep 09 23:23:10.213432 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.213402 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e42903f1-61a7-44b9-8062-8ee4e1b138b0-tls-certs\") pod \"model-serving-api-54d89bd5fc-9nx9s\" (UID: \"e42903f1-61a7-44b9-8062-8ee4e1b138b0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" Sep 09 23:23:10.218494 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.218470 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfqtw\" (UniqueName: \"kubernetes.io/projected/e42903f1-61a7-44b9-8062-8ee4e1b138b0-kube-api-access-cfqtw\") pod \"model-serving-api-54d89bd5fc-9nx9s\" (UID: \"e42903f1-61a7-44b9-8062-8ee4e1b138b0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" Sep 09 23:23:10.298893 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.298857 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 09 23:23:10.299011 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.298931 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 09 23:23:10.299011 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.298960 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 09 23:23:10.318843 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.318809 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" Sep 09 23:23:10.370454 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.370378 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" Sep 09 23:23:10.590499 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.590444 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9nx9s"] Sep 09 23:23:10.592547 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:23:10.592522 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42903f1_61a7_44b9_8062_8ee4e1b138b0.slice/crio-4d45c62766336c099ef021be317b88bd04227d48746edc307091a812c8fb2bec WatchSource:0}: Error finding container 4d45c62766336c099ef021be317b88bd04227d48746edc307091a812c8fb2bec: Status 404 returned error can't find the container with id 4d45c62766336c099ef021be317b88bd04227d48746edc307091a812c8fb2bec Sep 09 23:23:10.650865 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.650832 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nmx4g"] Sep 09 23:23:10.653637 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:23:10.653597 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf594896f_45b1_477d_8e1a_419edd1bc2c6.slice/crio-48244313a62428f79230d9c1e324e955f8c58ae1cab9e79ffc194acfd6e4467c WatchSource:0}: Error finding container 48244313a62428f79230d9c1e324e955f8c58ae1cab9e79ffc194acfd6e4467c: Status 404 returned error can't find the container with id 48244313a62428f79230d9c1e324e955f8c58ae1cab9e79ffc194acfd6e4467c Sep 09 23:23:10.683709 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.683673 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" event={"ID":"e42903f1-61a7-44b9-8062-8ee4e1b138b0","Type":"ContainerStarted","Data":"4d45c62766336c099ef021be317b88bd04227d48746edc307091a812c8fb2bec"} Sep 09 23:23:10.684946 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.684918 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" event={"ID":"f594896f-45b1-477d-8e1a-419edd1bc2c6","Type":"ContainerStarted","Data":"48244313a62428f79230d9c1e324e955f8c58ae1cab9e79ffc194acfd6e4467c"} Sep 09 23:23:10.686462 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.686436 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" event={"ID":"35ec7208-3462-40cc-8a7b-816a5d860510","Type":"ContainerStarted","Data":"cabc3529fa833af1644bf3b089ec7f17326db11e8c82b0af769712a841c937bb"} Sep 09 23:23:10.703712 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.703670 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" podStartSLOduration=1.67621056 podStartE2EDuration="4.703654305s" totalImagesPullingTime="3.027443745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:23:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:23:10.702275598 +0000 UTC m=+352.128698905" watchObservedRunningTime="2026-09-09 23:23:10.703654305 +0000 UTC m=+352.130077683" Sep 09 23:23:10.993049 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:10.992898 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:12.000669 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:12.000631 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:12.697982 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:12.697938 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:12.699019 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:12.698995 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5" Sep 09 23:23:13.297662 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:13.297625 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:13.298144 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:13.297688 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:13.303561 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:13.303529 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:13.706546 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:13.706469 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f46cd58f7-ds2nf" Sep 09 23:23:13.749884 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:13.749855 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c89f48688-wkqxd"] Sep 09 23:23:14.708484 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:14.708452 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" event={"ID":"e42903f1-61a7-44b9-8062-8ee4e1b138b0","Type":"ContainerStarted","Data":"3eedf4f96adfbdb4502e8fb010707150d60f0ddc243900e3f0f6a13806942749"} Sep 09 23:23:14.708945 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:14.708604 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" Sep 09 23:23:14.710022 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:14.709993 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" event={"ID":"f594896f-45b1-477d-8e1a-419edd1bc2c6","Type":"ContainerStarted","Data":"a6ea239f0b188e2b5214e4f3f96e8a034216def18913d1ae688973506ae22da7"} Sep 09 23:23:14.710386 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:14.710361 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" Sep 09 23:23:14.725508 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:14.725471 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" podStartSLOduration=2.562258128 podStartE2EDuration="5.725457951s" totalImagesPullingTime="3.163199823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:23:14.725103312 +0000 UTC m=+356.151526618" watchObservedRunningTime="2026-09-09 23:23:14.725457951 +0000 UTC m=+356.151881257" Sep 09 23:23:14.741448 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:14.741392 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" podStartSLOduration=2.634245937 podStartE2EDuration="5.741372802s" totalImagesPullingTime="3.107126865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:23:14.740415646 +0000 UTC m=+356.166838953" watchObservedRunningTime="2026-09-09 23:23:14.741372802 +0000 UTC m=+356.167796110" Sep 09 23:23:16.070750 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.070716 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf"] Sep 09 23:23:16.077095 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.077078 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.082296 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.082272 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 09 23:23:16.082424 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.082382 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 09 23:23:16.082424 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.082393 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fdckl\"" Sep 09 23:23:16.082519 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.082497 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 09 23:23:16.094412 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.094388 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf"] Sep 09 23:23:16.169925 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.169892 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c863aba1-e01d-4350-ae08-419d0a04e4d6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.170138 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.169939 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9kxp\" (UniqueName: \"kubernetes.io/projected/c863aba1-e01d-4350-ae08-419d0a04e4d6-kube-api-access-z9kxp\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.170138 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.169959 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c863aba1-e01d-4350-ae08-419d0a04e4d6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.170138 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.170084 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c863aba1-e01d-4350-ae08-419d0a04e4d6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.170138 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.170118 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c863aba1-e01d-4350-ae08-419d0a04e4d6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.170350 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.170201 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c863aba1-e01d-4350-ae08-419d0a04e4d6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.170350 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.170238 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c863aba1-e01d-4350-ae08-419d0a04e4d6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.271431 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.271384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c863aba1-e01d-4350-ae08-419d0a04e4d6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.271431 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.271430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c863aba1-e01d-4350-ae08-419d0a04e4d6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.271624 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.271569 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c863aba1-e01d-4350-ae08-419d0a04e4d6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.271624 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.271610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c863aba1-e01d-4350-ae08-419d0a04e4d6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.271689 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.271653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c863aba1-e01d-4350-ae08-419d0a04e4d6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.271756 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.271736 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9kxp\" (UniqueName: \"kubernetes.io/projected/c863aba1-e01d-4350-ae08-419d0a04e4d6-kube-api-access-z9kxp\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.271830 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.271776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c863aba1-e01d-4350-ae08-419d0a04e4d6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.272504 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.272472 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c863aba1-e01d-4350-ae08-419d0a04e4d6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.273848 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.273819 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c863aba1-e01d-4350-ae08-419d0a04e4d6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.273999 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.273960 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c863aba1-e01d-4350-ae08-419d0a04e4d6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.274085 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.274069 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c863aba1-e01d-4350-ae08-419d0a04e4d6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.274218 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.274202 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c863aba1-e01d-4350-ae08-419d0a04e4d6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.279510 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.279483 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c863aba1-e01d-4350-ae08-419d0a04e4d6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.279802 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.279785 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9kxp\" (UniqueName: \"kubernetes.io/projected/c863aba1-e01d-4350-ae08-419d0a04e4d6-kube-api-access-z9kxp\") pod \"istiod-openshift-gateway-5677ccb49b-zx8jf\" (UID: \"c863aba1-e01d-4350-ae08-419d0a04e4d6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.386081 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.386000 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:16.661016 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.660961 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf"] Sep 09 23:23:16.663321 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:23:16.663287 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc863aba1_e01d_4350_ae08_419d0a04e4d6.slice/crio-84c1a1949e01c0b29fb82c3359b4d7eea72c975c08ce0bf3e4891edbc59fe2ae WatchSource:0}: Error finding container 84c1a1949e01c0b29fb82c3359b4d7eea72c975c08ce0bf3e4891edbc59fe2ae: Status 404 returned error can't find the container with id 84c1a1949e01c0b29fb82c3359b4d7eea72c975c08ce0bf3e4891edbc59fe2ae Sep 09 23:23:16.720949 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:16.720911 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" event={"ID":"c863aba1-e01d-4350-ae08-419d0a04e4d6","Type":"ContainerStarted","Data":"84c1a1949e01c0b29fb82c3359b4d7eea72c975c08ce0bf3e4891edbc59fe2ae"} Sep 09 23:23:18.742662 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:18.742632 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hd246"] Sep 09 23:23:18.747547 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:18.747521 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" Sep 09 23:23:18.750545 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:18.750524 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4lw9m\"" Sep 09 23:23:18.750847 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:18.750829 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 09 23:23:18.765318 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:18.765231 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hd246"] Sep 09 23:23:18.894171 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:18.894125 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfjn9\" (UniqueName: \"kubernetes.io/projected/bbf2d253-13d6-483a-a536-cc7525f2b3e3-kube-api-access-lfjn9\") pod \"kserve-controller-manager-d6f4b65b8-hd246\" (UID: \"bbf2d253-13d6-483a-a536-cc7525f2b3e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" Sep 09 23:23:18.894367 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:18.894187 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbf2d253-13d6-483a-a536-cc7525f2b3e3-cert\") pod \"kserve-controller-manager-d6f4b65b8-hd246\" (UID: \"bbf2d253-13d6-483a-a536-cc7525f2b3e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" Sep 09 23:23:18.995226 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:18.995147 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfjn9\" (UniqueName: \"kubernetes.io/projected/bbf2d253-13d6-483a-a536-cc7525f2b3e3-kube-api-access-lfjn9\") pod \"kserve-controller-manager-d6f4b65b8-hd246\" (UID: \"bbf2d253-13d6-483a-a536-cc7525f2b3e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" Sep 09 23:23:18.995226 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:18.995186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbf2d253-13d6-483a-a536-cc7525f2b3e3-cert\") pod \"kserve-controller-manager-d6f4b65b8-hd246\" (UID: \"bbf2d253-13d6-483a-a536-cc7525f2b3e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" Sep 09 23:23:18.997763 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:18.997737 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 09 23:23:19.003808 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:19.003781 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfjn9\" (UniqueName: \"kubernetes.io/projected/bbf2d253-13d6-483a-a536-cc7525f2b3e3-kube-api-access-lfjn9\") pod \"kserve-controller-manager-d6f4b65b8-hd246\" (UID: \"bbf2d253-13d6-483a-a536-cc7525f2b3e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" Sep 09 23:23:19.008406 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:23:19.008130 2580 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Sep 09 23:23:19.008406 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:23:19.008211 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbf2d253-13d6-483a-a536-cc7525f2b3e3-cert podName:bbf2d253-13d6-483a-a536-cc7525f2b3e3 nodeName:}" failed. No retries permitted until 2026-09-09 23:23:19.508186239 +0000 UTC m=+360.934609538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bbf2d253-13d6-483a-a536-cc7525f2b3e3-cert") pod "kserve-controller-manager-d6f4b65b8-hd246" (UID: "bbf2d253-13d6-483a-a536-cc7525f2b3e3") : secret "kserve-webhook-server-cert" not found Sep 09 23:23:19.103377 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:19.103331 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 09 23:23:19.103670 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:19.103417 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 09 23:23:19.601078 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:19.601030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbf2d253-13d6-483a-a536-cc7525f2b3e3-cert\") pod \"kserve-controller-manager-d6f4b65b8-hd246\" (UID: \"bbf2d253-13d6-483a-a536-cc7525f2b3e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" Sep 09 23:23:19.603562 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:19.603535 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbf2d253-13d6-483a-a536-cc7525f2b3e3-cert\") pod \"kserve-controller-manager-d6f4b65b8-hd246\" (UID: \"bbf2d253-13d6-483a-a536-cc7525f2b3e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" Sep 09 23:23:19.662446 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:19.662414 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4lw9m\"" Sep 09 23:23:19.670100 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:19.670073 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" Sep 09 23:23:19.740918 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:19.740883 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" event={"ID":"c863aba1-e01d-4350-ae08-419d0a04e4d6","Type":"ContainerStarted","Data":"d43b2237588473f9ec5d7efad58c0432e3cf461d06142a1cd2a40129508b4f94"} Sep 09 23:23:19.741088 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:19.741006 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:19.795200 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:19.795143 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" podStartSLOduration=1.357959476 podStartE2EDuration="3.795128684s" totalImagesPullingTime="2.437169208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:23:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:23:19.794257294 +0000 UTC m=+361.220680613" watchObservedRunningTime="2026-09-09 23:23:19.795128684 +0000 UTC m=+361.221551990" Sep 09 23:23:19.944346 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:19.944321 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hd246"] Sep 09 23:23:19.946151 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:23:19.946124 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf2d253_13d6_483a_a536_cc7525f2b3e3.slice/crio-740a5e77bb17e361c4f049d4366da3a5cf24817ff05fc6eef2459e680f2818a8 WatchSource:0}: Error finding container 740a5e77bb17e361c4f049d4366da3a5cf24817ff05fc6eef2459e680f2818a8: Status 404 returned error can't find the container with id 740a5e77bb17e361c4f049d4366da3a5cf24817ff05fc6eef2459e680f2818a8 Sep 09 23:23:20.748133 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:20.748095 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" event={"ID":"bbf2d253-13d6-483a-a536-cc7525f2b3e3","Type":"ContainerStarted","Data":"740a5e77bb17e361c4f049d4366da3a5cf24817ff05fc6eef2459e680f2818a8"} Sep 09 23:23:20.750279 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:20.750251 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zx8jf" Sep 09 23:23:22.759425 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:22.759390 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" event={"ID":"bbf2d253-13d6-483a-a536-cc7525f2b3e3","Type":"ContainerStarted","Data":"544e990fd7f1aa86e58cd113ca0a0806ee090c08a1c9b7b28332a05098351fa7"} Sep 09 23:23:22.759817 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:22.759482 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" Sep 09 23:23:22.775905 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:22.775859 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" podStartSLOduration=2.61420763 podStartE2EDuration="4.775846819s" totalImagesPullingTime="2.161639189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:23:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:23:22.774329752 +0000 UTC m=+364.200753058" watchObservedRunningTime="2026-09-09 23:23:22.775846819 +0000 UTC m=+364.202270125" Sep 09 23:23:25.719260 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:25.719227 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9nx9s" Sep 09 23:23:26.723953 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:26.723920 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-nmx4g" Sep 09 23:23:38.778636 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:38.778591 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c89f48688-wkqxd" podUID="038ede4e-b6f5-4a6a-85e7-29f0e4319524" containerName="console" containerID="cri-o://b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed" gracePeriod=15 Sep 09 23:23:39.019088 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.019064 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:23:39.169653 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.169623 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-serving-cert\") pod \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " Sep 09 23:23:39.169818 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.169671 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-config\") pod \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " Sep 09 23:23:39.169818 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.169701 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp8jw\" (UniqueName: \"kubernetes.io/projected/038ede4e-b6f5-4a6a-85e7-29f0e4319524-kube-api-access-sp8jw\") pod \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " Sep 09 23:23:39.169818 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.169740 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-trusted-ca-bundle\") pod \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " Sep 09 23:23:39.169818 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.169758 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-oauth-config\") pod \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " Sep 09 23:23:39.169818 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.169774 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-service-ca\") pod \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " Sep 09 23:23:39.169818 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.169804 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-oauth-serving-cert\") pod \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\" (UID: \"038ede4e-b6f5-4a6a-85e7-29f0e4319524\") " Sep 09 23:23:39.170271 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.170236 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "038ede4e-b6f5-4a6a-85e7-29f0e4319524" (UID: "038ede4e-b6f5-4a6a-85e7-29f0e4319524"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:23:39.170373 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.170347 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-service-ca" (OuterVolumeSpecName: "service-ca") pod "038ede4e-b6f5-4a6a-85e7-29f0e4319524" (UID: "038ede4e-b6f5-4a6a-85e7-29f0e4319524"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:23:39.170441 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.170411 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "038ede4e-b6f5-4a6a-85e7-29f0e4319524" (UID: "038ede4e-b6f5-4a6a-85e7-29f0e4319524"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:23:39.170589 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.170564 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-config" (OuterVolumeSpecName: "console-config") pod "038ede4e-b6f5-4a6a-85e7-29f0e4319524" (UID: "038ede4e-b6f5-4a6a-85e7-29f0e4319524"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:23:39.171869 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.171842 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "038ede4e-b6f5-4a6a-85e7-29f0e4319524" (UID: "038ede4e-b6f5-4a6a-85e7-29f0e4319524"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:23:39.172001 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.171936 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/038ede4e-b6f5-4a6a-85e7-29f0e4319524-kube-api-access-sp8jw" (OuterVolumeSpecName: "kube-api-access-sp8jw") pod "038ede4e-b6f5-4a6a-85e7-29f0e4319524" (UID: "038ede4e-b6f5-4a6a-85e7-29f0e4319524"). InnerVolumeSpecName "kube-api-access-sp8jw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:23:39.172001 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.171981 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "038ede4e-b6f5-4a6a-85e7-29f0e4319524" (UID: "038ede4e-b6f5-4a6a-85e7-29f0e4319524"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:23:39.270512 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.270470 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-config\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:23:39.270512 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.270506 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp8jw\" (UniqueName: \"kubernetes.io/projected/038ede4e-b6f5-4a6a-85e7-29f0e4319524-kube-api-access-sp8jw\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:23:39.270512 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.270521 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-trusted-ca-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:23:39.270760 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.270533 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-oauth-config\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:23:39.270760 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.270545 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-service-ca\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:23:39.270760 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.270556 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/038ede4e-b6f5-4a6a-85e7-29f0e4319524-oauth-serving-cert\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:23:39.270760 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.270567 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/038ede4e-b6f5-4a6a-85e7-29f0e4319524-console-serving-cert\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:23:39.848095 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.848056 2580 generic.go:356] "Generic (PLEG): container finished" podID="038ede4e-b6f5-4a6a-85e7-29f0e4319524" containerID="b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed" exitCode=0 Sep 09 23:23:39.848603 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.848112 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c89f48688-wkqxd" Sep 09 23:23:39.848603 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.848140 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c89f48688-wkqxd" event={"ID":"038ede4e-b6f5-4a6a-85e7-29f0e4319524","Type":"ContainerDied","Data":"b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed"} Sep 09 23:23:39.848603 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.848213 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c89f48688-wkqxd" event={"ID":"038ede4e-b6f5-4a6a-85e7-29f0e4319524","Type":"ContainerDied","Data":"70f0e4251a5db493b3a51dd42a6ece4d67db38eedc1a08920bdee5bea7514496"} Sep 09 23:23:39.848603 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.848234 2580 scope.go:117] "RemoveContainer" containerID="b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed" Sep 09 23:23:39.857410 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.857394 2580 scope.go:117] "RemoveContainer" containerID="b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed" Sep 09 23:23:39.857666 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:23:39.857649 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed\": container with ID starting with b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed not found: ID does not exist" containerID="b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed" Sep 09 23:23:39.857728 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.857676 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed"} err="failed to get container status \"b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed\": rpc error: code = NotFound desc = could not find container \"b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed\": container with ID starting with b9784f82fcd0322c19f9a704dbca9ebe401c268613d71a58d00a38a0c70864ed not found: ID does not exist" Sep 09 23:23:39.869741 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.869714 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c89f48688-wkqxd"] Sep 09 23:23:39.873709 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:39.873687 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c89f48688-wkqxd"] Sep 09 23:23:41.124235 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:41.124201 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="038ede4e-b6f5-4a6a-85e7-29f0e4319524" path="/var/lib/kubelet/pods/038ede4e-b6f5-4a6a-85e7-29f0e4319524/volumes" Sep 09 23:23:53.770927 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:23:53.770895 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hd246" Sep 09 23:24:13.981812 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:13.981778 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8"] Sep 09 23:24:13.982281 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:13.982135 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038ede4e-b6f5-4a6a-85e7-29f0e4319524" containerName="console" Sep 09 23:24:13.982281 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:13.982217 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="038ede4e-b6f5-4a6a-85e7-29f0e4319524" containerName="console" Sep 09 23:24:13.982360 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:13.982348 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="038ede4e-b6f5-4a6a-85e7-29f0e4319524" containerName="console" Sep 09 23:24:13.985634 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:13.985615 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:13.987978 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:13.987943 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 09 23:24:13.988083 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:13.987960 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 09 23:24:13.988870 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:13.988852 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bkm2k\"" Sep 09 23:24:13.992661 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:13.992639 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8"] Sep 09 23:24:14.052784 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.052749 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8\" (UID: \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:14.052784 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.052782 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8\" (UID: \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:14.053018 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.052839 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdx45\" (UniqueName: \"kubernetes.io/projected/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-kube-api-access-mdx45\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8\" (UID: \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:14.081179 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.081151 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj"] Sep 09 23:24:14.084523 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.084508 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:14.091843 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.091820 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj"] Sep 09 23:24:14.153479 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.153441 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d323f99-fbd7-48c4-831f-5211f22dce48-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj\" (UID: \"3d323f99-fbd7-48c4-831f-5211f22dce48\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:14.153672 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.153488 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8\" (UID: \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:14.153672 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.153512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d323f99-fbd7-48c4-831f-5211f22dce48-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj\" (UID: \"3d323f99-fbd7-48c4-831f-5211f22dce48\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:14.153672 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.153534 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8\" (UID: \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:14.153672 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.153637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdx45\" (UniqueName: \"kubernetes.io/projected/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-kube-api-access-mdx45\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8\" (UID: \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:14.153875 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.153757 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr4lj\" (UniqueName: \"kubernetes.io/projected/3d323f99-fbd7-48c4-831f-5211f22dce48-kube-api-access-hr4lj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj\" (UID: \"3d323f99-fbd7-48c4-831f-5211f22dce48\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:14.153914 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.153890 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8\" (UID: \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:14.153948 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.153913 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8\" (UID: \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:14.166062 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.166035 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdx45\" (UniqueName: \"kubernetes.io/projected/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-kube-api-access-mdx45\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8\" (UID: \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:14.214734 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.214705 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2"] Sep 09 23:24:14.218233 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.218218 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:14.226604 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.226584 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2"] Sep 09 23:24:14.254656 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.254582 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d323f99-fbd7-48c4-831f-5211f22dce48-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj\" (UID: \"3d323f99-fbd7-48c4-831f-5211f22dce48\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:14.254656 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.254621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d323f99-fbd7-48c4-831f-5211f22dce48-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj\" (UID: \"3d323f99-fbd7-48c4-831f-5211f22dce48\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:14.254812 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.254709 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr4lj\" (UniqueName: \"kubernetes.io/projected/3d323f99-fbd7-48c4-831f-5211f22dce48-kube-api-access-hr4lj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj\" (UID: \"3d323f99-fbd7-48c4-831f-5211f22dce48\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:14.255003 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.254962 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d323f99-fbd7-48c4-831f-5211f22dce48-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj\" (UID: \"3d323f99-fbd7-48c4-831f-5211f22dce48\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:14.255049 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.254987 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d323f99-fbd7-48c4-831f-5211f22dce48-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj\" (UID: \"3d323f99-fbd7-48c4-831f-5211f22dce48\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:14.263713 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.263683 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr4lj\" (UniqueName: \"kubernetes.io/projected/3d323f99-fbd7-48c4-831f-5211f22dce48-kube-api-access-hr4lj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj\" (UID: \"3d323f99-fbd7-48c4-831f-5211f22dce48\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:14.280597 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.280564 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw"] Sep 09 23:24:14.284061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.284041 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:14.292291 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.292260 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw"] Sep 09 23:24:14.295856 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.295836 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:14.355627 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.355562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2\" (UID: \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:14.355794 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.355649 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td2kq\" (UniqueName: \"kubernetes.io/projected/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-kube-api-access-td2kq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2\" (UID: \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:14.355794 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.355700 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw\" (UID: \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:14.355794 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.355766 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2\" (UID: \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:14.356034 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.355795 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw\" (UID: \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:14.356034 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.355818 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhgz4\" (UniqueName: \"kubernetes.io/projected/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-kube-api-access-nhgz4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw\" (UID: \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:14.393418 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.393386 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:14.456554 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.456521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2\" (UID: \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:14.456703 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.456560 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw\" (UID: \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:14.456703 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.456579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhgz4\" (UniqueName: \"kubernetes.io/projected/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-kube-api-access-nhgz4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw\" (UID: \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:14.456703 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.456623 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2\" (UID: \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:14.456703 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.456664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td2kq\" (UniqueName: \"kubernetes.io/projected/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-kube-api-access-td2kq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2\" (UID: \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:14.456933 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.456783 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw\" (UID: \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:14.457010 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.456996 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw\" (UID: \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:14.457074 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.456996 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2\" (UID: \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:14.457074 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.457047 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2\" (UID: \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:14.457316 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.457296 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw\" (UID: \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:14.465027 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.464955 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td2kq\" (UniqueName: \"kubernetes.io/projected/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-kube-api-access-td2kq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2\" (UID: \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:14.465603 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.465573 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhgz4\" (UniqueName: \"kubernetes.io/projected/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-kube-api-access-nhgz4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw\" (UID: \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:14.528305 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.528226 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:14.570655 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.570626 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8"] Sep 09 23:24:14.572827 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:24:14.572799 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda29bf5c5_b6c6_4b69_8d34_837d627ac5c2.slice/crio-b82a443bf698b1e9f9596df080759e66720f8332f097b6c321933cba046ee616 WatchSource:0}: Error finding container b82a443bf698b1e9f9596df080759e66720f8332f097b6c321933cba046ee616: Status 404 returned error can't find the container with id b82a443bf698b1e9f9596df080759e66720f8332f097b6c321933cba046ee616 Sep 09 23:24:14.594368 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.594342 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:14.664476 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.664447 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj"] Sep 09 23:24:14.666507 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:24:14.666478 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d323f99_fbd7_48c4_831f_5211f22dce48.slice/crio-4538639ac26a56b87ff330085c733deebd8611d1edace09fc4f45043cd11f20f WatchSource:0}: Error finding container 4538639ac26a56b87ff330085c733deebd8611d1edace09fc4f45043cd11f20f: Status 404 returned error can't find the container with id 4538639ac26a56b87ff330085c733deebd8611d1edace09fc4f45043cd11f20f Sep 09 23:24:14.801114 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.801089 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2"] Sep 09 23:24:14.837728 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:24:14.837696 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef8e62d_2d13_4142_b1d8_54bbafa7e108.slice/crio-9c2be41f865682d1807381a57785aa19c120339e68a20b5c362819a48dcc1ac6 WatchSource:0}: Error finding container 9c2be41f865682d1807381a57785aa19c120339e68a20b5c362819a48dcc1ac6: Status 404 returned error can't find the container with id 9c2be41f865682d1807381a57785aa19c120339e68a20b5c362819a48dcc1ac6 Sep 09 23:24:14.869801 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:14.869772 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw"] Sep 09 23:24:14.870172 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:24:14.870149 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3aa371_43b4_4ab2_8fee_4e0cee5d3307.slice/crio-c6272198eb62431a789f503e3573e14c357dc80a1ef68dc10d17fb7e33bccf2a WatchSource:0}: Error finding container c6272198eb62431a789f503e3573e14c357dc80a1ef68dc10d17fb7e33bccf2a: Status 404 returned error can't find the container with id c6272198eb62431a789f503e3573e14c357dc80a1ef68dc10d17fb7e33bccf2a Sep 09 23:24:15.029210 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:15.029175 2580 generic.go:356] "Generic (PLEG): container finished" podID="a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" containerID="55438bdecd6c2e6151502ab9c920e570f8153ba289a28f6f41bfffa4b80ccc96" exitCode=0 Sep 09 23:24:15.029593 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:15.029264 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" event={"ID":"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2","Type":"ContainerDied","Data":"55438bdecd6c2e6151502ab9c920e570f8153ba289a28f6f41bfffa4b80ccc96"} Sep 09 23:24:15.029593 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:15.029320 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" event={"ID":"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2","Type":"ContainerStarted","Data":"b82a443bf698b1e9f9596df080759e66720f8332f097b6c321933cba046ee616"} Sep 09 23:24:15.030865 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:15.030842 2580 generic.go:356] "Generic (PLEG): container finished" podID="1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" containerID="f0d281565471b44b2e53831e7b9babf22323505b7932cf17b804d282e27e26ff" exitCode=0 Sep 09 23:24:15.031175 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:15.030930 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" event={"ID":"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307","Type":"ContainerDied","Data":"f0d281565471b44b2e53831e7b9babf22323505b7932cf17b804d282e27e26ff"} Sep 09 23:24:15.031175 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:15.031009 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" event={"ID":"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307","Type":"ContainerStarted","Data":"c6272198eb62431a789f503e3573e14c357dc80a1ef68dc10d17fb7e33bccf2a"} Sep 09 23:24:15.032751 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:15.032731 2580 generic.go:356] "Generic (PLEG): container finished" podID="3d323f99-fbd7-48c4-831f-5211f22dce48" containerID="aed697e30d79dccac58c3220a78f220887a2e3eff5139098ce1a30f027da97eb" exitCode=0 Sep 09 23:24:15.032838 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:15.032818 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" event={"ID":"3d323f99-fbd7-48c4-831f-5211f22dce48","Type":"ContainerDied","Data":"aed697e30d79dccac58c3220a78f220887a2e3eff5139098ce1a30f027da97eb"} Sep 09 23:24:15.032896 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:15.032865 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" event={"ID":"3d323f99-fbd7-48c4-831f-5211f22dce48","Type":"ContainerStarted","Data":"4538639ac26a56b87ff330085c733deebd8611d1edace09fc4f45043cd11f20f"} Sep 09 23:24:15.034479 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:15.034462 2580 generic.go:356] "Generic (PLEG): container finished" podID="9ef8e62d-2d13-4142-b1d8-54bbafa7e108" containerID="fecd890699d93c5dabb8cca63e215567da8106b0d5fd494994b95cd4e460bb96" exitCode=0 Sep 09 23:24:15.034560 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:15.034501 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" event={"ID":"9ef8e62d-2d13-4142-b1d8-54bbafa7e108","Type":"ContainerDied","Data":"fecd890699d93c5dabb8cca63e215567da8106b0d5fd494994b95cd4e460bb96"} Sep 09 23:24:15.034560 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:15.034529 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" event={"ID":"9ef8e62d-2d13-4142-b1d8-54bbafa7e108","Type":"ContainerStarted","Data":"9c2be41f865682d1807381a57785aa19c120339e68a20b5c362819a48dcc1ac6"} Sep 09 23:24:17.047113 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:17.047078 2580 generic.go:356] "Generic (PLEG): container finished" podID="9ef8e62d-2d13-4142-b1d8-54bbafa7e108" containerID="ce40abc6506de108c4b2848bd4eba016dc666f3f02cb750618e70d5994d0d9b8" exitCode=0 Sep 09 23:24:17.047584 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:17.047161 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" event={"ID":"9ef8e62d-2d13-4142-b1d8-54bbafa7e108","Type":"ContainerDied","Data":"ce40abc6506de108c4b2848bd4eba016dc666f3f02cb750618e70d5994d0d9b8"} Sep 09 23:24:17.048948 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:17.048923 2580 generic.go:356] "Generic (PLEG): container finished" podID="a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" containerID="509fddcb54fc2abc0d7cfca4d101e5e597e6fb8a4230dc6490271c5e2cedf610" exitCode=0 Sep 09 23:24:17.049085 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:17.048947 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" event={"ID":"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2","Type":"ContainerDied","Data":"509fddcb54fc2abc0d7cfca4d101e5e597e6fb8a4230dc6490271c5e2cedf610"} Sep 09 23:24:17.050852 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:17.050829 2580 generic.go:356] "Generic (PLEG): container finished" podID="1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" containerID="6d7e198f3e5b3013688334885c6e1946186d7028b35a51573737d40f55713999" exitCode=0 Sep 09 23:24:17.050945 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:17.050895 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" event={"ID":"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307","Type":"ContainerDied","Data":"6d7e198f3e5b3013688334885c6e1946186d7028b35a51573737d40f55713999"} Sep 09 23:24:17.052801 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:17.052775 2580 generic.go:356] "Generic (PLEG): container finished" podID="3d323f99-fbd7-48c4-831f-5211f22dce48" containerID="2b6e7c78a809b91297bcd9a63a0e2e1da78bd8ea8e9fca63635664120b16aad6" exitCode=0 Sep 09 23:24:17.052850 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:17.052816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" event={"ID":"3d323f99-fbd7-48c4-831f-5211f22dce48","Type":"ContainerDied","Data":"2b6e7c78a809b91297bcd9a63a0e2e1da78bd8ea8e9fca63635664120b16aad6"} Sep 09 23:24:18.059809 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:18.059774 2580 generic.go:356] "Generic (PLEG): container finished" podID="9ef8e62d-2d13-4142-b1d8-54bbafa7e108" containerID="fd5f8cd02e74f05ce8617f9593aa19973a7aa997a5d84be8581c7fdf223c57b0" exitCode=0 Sep 09 23:24:18.060266 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:18.059864 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" event={"ID":"9ef8e62d-2d13-4142-b1d8-54bbafa7e108","Type":"ContainerDied","Data":"fd5f8cd02e74f05ce8617f9593aa19973a7aa997a5d84be8581c7fdf223c57b0"} Sep 09 23:24:18.061938 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:18.061914 2580 generic.go:356] "Generic (PLEG): container finished" podID="a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" containerID="f6cc6597b7b294c03b07fa564073f5284631394937c163b2d082a9870bcf5dcb" exitCode=0 Sep 09 23:24:18.062061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:18.062000 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" event={"ID":"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2","Type":"ContainerDied","Data":"f6cc6597b7b294c03b07fa564073f5284631394937c163b2d082a9870bcf5dcb"} Sep 09 23:24:18.064076 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:18.064056 2580 generic.go:356] "Generic (PLEG): container finished" podID="1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" containerID="4487a8be69530f0c1024c5acb4bad390683e40a08ce9fd5df6a7fd6a2e6bd591" exitCode=0 Sep 09 23:24:18.064194 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:18.064119 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" event={"ID":"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307","Type":"ContainerDied","Data":"4487a8be69530f0c1024c5acb4bad390683e40a08ce9fd5df6a7fd6a2e6bd591"} Sep 09 23:24:18.066033 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:18.066015 2580 generic.go:356] "Generic (PLEG): container finished" podID="3d323f99-fbd7-48c4-831f-5211f22dce48" containerID="70aca352fdd34f0354642e00a66073c042e275a51f4b9d296aec7f83f15390d1" exitCode=0 Sep 09 23:24:18.066112 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:18.066051 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" event={"ID":"3d323f99-fbd7-48c4-831f-5211f22dce48","Type":"ContainerDied","Data":"70aca352fdd34f0354642e00a66073c042e275a51f4b9d296aec7f83f15390d1"} Sep 09 23:24:19.226889 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.226855 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:19.268312 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.268288 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:19.271172 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.271156 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:19.274451 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.274431 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:19.312524 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.312489 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-bundle\") pod \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\" (UID: \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\") " Sep 09 23:24:19.312676 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.312640 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-util\") pod \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\" (UID: \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\") " Sep 09 23:24:19.312736 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.312683 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td2kq\" (UniqueName: \"kubernetes.io/projected/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-kube-api-access-td2kq\") pod \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\" (UID: \"9ef8e62d-2d13-4142-b1d8-54bbafa7e108\") " Sep 09 23:24:19.313415 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.313373 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-bundle" (OuterVolumeSpecName: "bundle") pod "9ef8e62d-2d13-4142-b1d8-54bbafa7e108" (UID: "9ef8e62d-2d13-4142-b1d8-54bbafa7e108"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:24:19.315701 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.315670 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-kube-api-access-td2kq" (OuterVolumeSpecName: "kube-api-access-td2kq") pod "9ef8e62d-2d13-4142-b1d8-54bbafa7e108" (UID: "9ef8e62d-2d13-4142-b1d8-54bbafa7e108"). InnerVolumeSpecName "kube-api-access-td2kq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:24:19.318723 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.318696 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-util" (OuterVolumeSpecName: "util") pod "9ef8e62d-2d13-4142-b1d8-54bbafa7e108" (UID: "9ef8e62d-2d13-4142-b1d8-54bbafa7e108"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:24:19.413952 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.413863 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr4lj\" (UniqueName: \"kubernetes.io/projected/3d323f99-fbd7-48c4-831f-5211f22dce48-kube-api-access-hr4lj\") pod \"3d323f99-fbd7-48c4-831f-5211f22dce48\" (UID: \"3d323f99-fbd7-48c4-831f-5211f22dce48\") " Sep 09 23:24:19.413952 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.413908 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-bundle\") pod \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\" (UID: \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\") " Sep 09 23:24:19.413952 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.413926 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-bundle\") pod \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\" (UID: \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\") " Sep 09 23:24:19.413952 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.413953 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-util\") pod \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\" (UID: \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\") " Sep 09 23:24:19.414300 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.414006 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d323f99-fbd7-48c4-831f-5211f22dce48-util\") pod \"3d323f99-fbd7-48c4-831f-5211f22dce48\" (UID: \"3d323f99-fbd7-48c4-831f-5211f22dce48\") " Sep 09 23:24:19.414300 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.414048 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhgz4\" (UniqueName: \"kubernetes.io/projected/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-kube-api-access-nhgz4\") pod \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\" (UID: \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\") " Sep 09 23:24:19.414300 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.414081 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-util\") pod \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\" (UID: \"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307\") " Sep 09 23:24:19.414300 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.414115 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdx45\" (UniqueName: \"kubernetes.io/projected/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-kube-api-access-mdx45\") pod \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\" (UID: \"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2\") " Sep 09 23:24:19.414300 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.414194 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d323f99-fbd7-48c4-831f-5211f22dce48-bundle\") pod \"3d323f99-fbd7-48c4-831f-5211f22dce48\" (UID: \"3d323f99-fbd7-48c4-831f-5211f22dce48\") " Sep 09 23:24:19.414561 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.414479 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:19.414561 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.414496 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-util\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:19.414561 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.414509 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td2kq\" (UniqueName: \"kubernetes.io/projected/9ef8e62d-2d13-4142-b1d8-54bbafa7e108-kube-api-access-td2kq\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:19.414561 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.414540 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-bundle" (OuterVolumeSpecName: "bundle") pod "1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" (UID: "1f3aa371-43b4-4ab2-8fee-4e0cee5d3307"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:24:19.415172 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.414951 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-bundle" (OuterVolumeSpecName: "bundle") pod "a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" (UID: "a29bf5c5-b6c6-4b69-8d34-837d627ac5c2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:24:19.415172 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.415128 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d323f99-fbd7-48c4-831f-5211f22dce48-bundle" (OuterVolumeSpecName: "bundle") pod "3d323f99-fbd7-48c4-831f-5211f22dce48" (UID: "3d323f99-fbd7-48c4-831f-5211f22dce48"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:24:19.416403 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.416377 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d323f99-fbd7-48c4-831f-5211f22dce48-kube-api-access-hr4lj" (OuterVolumeSpecName: "kube-api-access-hr4lj") pod "3d323f99-fbd7-48c4-831f-5211f22dce48" (UID: "3d323f99-fbd7-48c4-831f-5211f22dce48"). InnerVolumeSpecName "kube-api-access-hr4lj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:24:19.417030 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.417012 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-kube-api-access-mdx45" (OuterVolumeSpecName: "kube-api-access-mdx45") pod "a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" (UID: "a29bf5c5-b6c6-4b69-8d34-837d627ac5c2"). InnerVolumeSpecName "kube-api-access-mdx45". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:24:19.417512 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.417493 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-kube-api-access-nhgz4" (OuterVolumeSpecName: "kube-api-access-nhgz4") pod "1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" (UID: "1f3aa371-43b4-4ab2-8fee-4e0cee5d3307"). InnerVolumeSpecName "kube-api-access-nhgz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:24:19.420368 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.420321 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d323f99-fbd7-48c4-831f-5211f22dce48-util" (OuterVolumeSpecName: "util") pod "3d323f99-fbd7-48c4-831f-5211f22dce48" (UID: "3d323f99-fbd7-48c4-831f-5211f22dce48"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:24:19.420891 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.420870 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-util" (OuterVolumeSpecName: "util") pod "a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" (UID: "a29bf5c5-b6c6-4b69-8d34-837d627ac5c2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:24:19.421149 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.421124 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-util" (OuterVolumeSpecName: "util") pod "1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" (UID: "1f3aa371-43b4-4ab2-8fee-4e0cee5d3307"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:24:19.515322 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.515287 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-util\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:19.515322 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.515314 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d323f99-fbd7-48c4-831f-5211f22dce48-util\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:19.515322 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.515327 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhgz4\" (UniqueName: \"kubernetes.io/projected/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-kube-api-access-nhgz4\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:19.515545 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.515340 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-util\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:19.515545 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.515353 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdx45\" (UniqueName: \"kubernetes.io/projected/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-kube-api-access-mdx45\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:19.515545 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.515365 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d323f99-fbd7-48c4-831f-5211f22dce48-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:19.515545 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.515376 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr4lj\" (UniqueName: \"kubernetes.io/projected/3d323f99-fbd7-48c4-831f-5211f22dce48-kube-api-access-hr4lj\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:19.515545 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.515387 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f3aa371-43b4-4ab2-8fee-4e0cee5d3307-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:19.515545 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:19.515398 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a29bf5c5-b6c6-4b69-8d34-837d627ac5c2-bundle\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:20.078041 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:20.078015 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" Sep 09 23:24:20.078196 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:20.078006 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vhqt2" event={"ID":"9ef8e62d-2d13-4142-b1d8-54bbafa7e108","Type":"ContainerDied","Data":"9c2be41f865682d1807381a57785aa19c120339e68a20b5c362819a48dcc1ac6"} Sep 09 23:24:20.078245 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:20.078191 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c2be41f865682d1807381a57785aa19c120339e68a20b5c362819a48dcc1ac6" Sep 09 23:24:20.079908 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:20.079889 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" Sep 09 23:24:20.080055 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:20.079914 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7lrf8" event={"ID":"a29bf5c5-b6c6-4b69-8d34-837d627ac5c2","Type":"ContainerDied","Data":"b82a443bf698b1e9f9596df080759e66720f8332f097b6c321933cba046ee616"} Sep 09 23:24:20.080055 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:20.079956 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b82a443bf698b1e9f9596df080759e66720f8332f097b6c321933cba046ee616" Sep 09 23:24:20.081810 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:20.081788 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" event={"ID":"1f3aa371-43b4-4ab2-8fee-4e0cee5d3307","Type":"ContainerDied","Data":"c6272198eb62431a789f503e3573e14c357dc80a1ef68dc10d17fb7e33bccf2a"} Sep 09 23:24:20.081900 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:20.081820 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6272198eb62431a789f503e3573e14c357dc80a1ef68dc10d17fb7e33bccf2a" Sep 09 23:24:20.081900 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:20.081828 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mbgbw" Sep 09 23:24:20.083759 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:20.083734 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" event={"ID":"3d323f99-fbd7-48c4-831f-5211f22dce48","Type":"ContainerDied","Data":"4538639ac26a56b87ff330085c733deebd8611d1edace09fc4f45043cd11f20f"} Sep 09 23:24:20.083861 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:20.083769 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4538639ac26a56b87ff330085c733deebd8611d1edace09fc4f45043cd11f20f" Sep 09 23:24:20.083861 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:20.083802 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz4zmj" Sep 09 23:24:26.460034 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.459999 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs"] Sep 09 23:24:26.460396 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460345 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef8e62d-2d13-4142-b1d8-54bbafa7e108" containerName="pull" Sep 09 23:24:26.460396 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460356 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef8e62d-2d13-4142-b1d8-54bbafa7e108" containerName="pull" Sep 09 23:24:26.460396 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460365 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d323f99-fbd7-48c4-831f-5211f22dce48" containerName="extract" Sep 09 23:24:26.460396 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460371 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d323f99-fbd7-48c4-831f-5211f22dce48" containerName="extract" Sep 09 23:24:26.460396 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460379 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" containerName="extract" Sep 09 23:24:26.460396 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460385 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" containerName="extract" Sep 09 23:24:26.460396 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460394 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" containerName="pull" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460401 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" containerName="pull" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460416 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef8e62d-2d13-4142-b1d8-54bbafa7e108" containerName="util" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460423 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef8e62d-2d13-4142-b1d8-54bbafa7e108" containerName="util" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460430 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" containerName="extract" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460436 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" containerName="extract" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460441 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef8e62d-2d13-4142-b1d8-54bbafa7e108" containerName="extract" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460447 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef8e62d-2d13-4142-b1d8-54bbafa7e108" containerName="extract" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460456 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" containerName="pull" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460461 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" containerName="pull" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460471 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" containerName="util" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460476 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" containerName="util" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460487 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d323f99-fbd7-48c4-831f-5211f22dce48" containerName="util" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460494 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d323f99-fbd7-48c4-831f-5211f22dce48" containerName="util" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460500 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" containerName="util" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460505 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" containerName="util" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460510 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d323f99-fbd7-48c4-831f-5211f22dce48" containerName="pull" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460515 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d323f99-fbd7-48c4-831f-5211f22dce48" containerName="pull" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460567 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a29bf5c5-b6c6-4b69-8d34-837d627ac5c2" containerName="extract" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460577 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d323f99-fbd7-48c4-831f-5211f22dce48" containerName="extract" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460584 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f3aa371-43b4-4ab2-8fee-4e0cee5d3307" containerName="extract" Sep 09 23:24:26.460625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.460591 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ef8e62d-2d13-4142-b1d8-54bbafa7e108" containerName="extract" Sep 09 23:24:26.464864 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.464846 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" Sep 09 23:24:26.467445 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.467420 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 09 23:24:26.467541 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.467460 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 09 23:24:26.468539 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.468523 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-855lf\"" Sep 09 23:24:26.473196 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.473166 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs"] Sep 09 23:24:26.580391 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.580346 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsb9x\" (UniqueName: \"kubernetes.io/projected/6f6eae0f-cf42-4f44-a17d-d6526d7441a8-kube-api-access-wsb9x\") pod \"limitador-operator-controller-manager-74495c69f8-ff4hs\" (UID: \"6f6eae0f-cf42-4f44-a17d-d6526d7441a8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" Sep 09 23:24:26.681669 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.681632 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsb9x\" (UniqueName: \"kubernetes.io/projected/6f6eae0f-cf42-4f44-a17d-d6526d7441a8-kube-api-access-wsb9x\") pod \"limitador-operator-controller-manager-74495c69f8-ff4hs\" (UID: \"6f6eae0f-cf42-4f44-a17d-d6526d7441a8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" Sep 09 23:24:26.699961 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.699923 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsb9x\" (UniqueName: \"kubernetes.io/projected/6f6eae0f-cf42-4f44-a17d-d6526d7441a8-kube-api-access-wsb9x\") pod \"limitador-operator-controller-manager-74495c69f8-ff4hs\" (UID: \"6f6eae0f-cf42-4f44-a17d-d6526d7441a8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" Sep 09 23:24:26.776787 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:26.776687 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" Sep 09 23:24:27.054317 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:27.054289 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs"] Sep 09 23:24:27.056841 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:24:27.056807 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f6eae0f_cf42_4f44_a17d_d6526d7441a8.slice/crio-512f78b274dc6f8ac1b4b681aeaf53dc90d1d457e4ab6abd6a17c35592063a95 WatchSource:0}: Error finding container 512f78b274dc6f8ac1b4b681aeaf53dc90d1d457e4ab6abd6a17c35592063a95: Status 404 returned error can't find the container with id 512f78b274dc6f8ac1b4b681aeaf53dc90d1d457e4ab6abd6a17c35592063a95 Sep 09 23:24:27.123692 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:27.123661 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" event={"ID":"6f6eae0f-cf42-4f44-a17d-d6526d7441a8","Type":"ContainerStarted","Data":"512f78b274dc6f8ac1b4b681aeaf53dc90d1d457e4ab6abd6a17c35592063a95"} Sep 09 23:24:30.142117 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:30.142079 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" event={"ID":"6f6eae0f-cf42-4f44-a17d-d6526d7441a8","Type":"ContainerStarted","Data":"51fb4d8529fec1b9babb3b3a6204dccd691c469a40d8944180686de586e98ef6"} Sep 09 23:24:30.142509 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:30.142260 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" Sep 09 23:24:30.159752 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:30.159706 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" podStartSLOduration=2.027638515 podStartE2EDuration="4.159693152s" totalImagesPullingTime="2.132054637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:24:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:24:30.158667105 +0000 UTC m=+431.585090423" watchObservedRunningTime="2026-09-09 23:24:30.159693152 +0000 UTC m=+431.586116460" Sep 09 23:24:41.148659 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:41.148576 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" Sep 09 23:24:49.015780 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.015741 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs"] Sep 09 23:24:49.016227 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.016001 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" podUID="6f6eae0f-cf42-4f44-a17d-d6526d7441a8" containerName="manager" containerID="cri-o://51fb4d8529fec1b9babb3b3a6204dccd691c469a40d8944180686de586e98ef6" gracePeriod=2 Sep 09 23:24:49.025282 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.025257 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs"] Sep 09 23:24:49.030661 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:24:49.030625 2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6f6eae0f-cf42-4f44-a17d-d6526d7441a8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-ff4hs\": pods \"limitador-operator-controller-manager-74495c69f8-ff4hs\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" Sep 09 23:24:49.037744 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.037722 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz"] Sep 09 23:24:49.038067 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.038054 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f6eae0f-cf42-4f44-a17d-d6526d7441a8" containerName="manager" Sep 09 23:24:49.038116 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.038069 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f6eae0f-cf42-4f44-a17d-d6526d7441a8" containerName="manager" Sep 09 23:24:49.038163 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.038123 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f6eae0f-cf42-4f44-a17d-d6526d7441a8" containerName="manager" Sep 09 23:24:49.039876 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.039862 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz" Sep 09 23:24:49.042087 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:24:49.042059 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ff4hs\" is forbidden: User \"system:node:ip-10-0-130-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-56.ec2.internal' and this object" podUID="6f6eae0f-cf42-4f44-a17d-d6526d7441a8" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" Sep 09 23:24:49.057364 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.057337 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz"] Sep 09 23:24:49.085307 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.085280 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmqrl\" (UniqueName: \"kubernetes.io/projected/8d9d6c34-100e-433a-a8cc-5a4628d08de6-kube-api-access-pmqrl\") pod \"limitador-operator-controller-manager-74495c69f8-r69dz\" (UID: \"8d9d6c34-100e-433a-a8cc-5a4628d08de6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz" Sep 09 23:24:49.124276 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:24:49.124241 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ff4hs\" is forbidden: User \"system:node:ip-10-0-130-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-56.ec2.internal' and this object" podUID="6f6eae0f-cf42-4f44-a17d-d6526d7441a8" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" Sep 09 23:24:49.186429 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.186391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmqrl\" (UniqueName: \"kubernetes.io/projected/8d9d6c34-100e-433a-a8cc-5a4628d08de6-kube-api-access-pmqrl\") pod \"limitador-operator-controller-manager-74495c69f8-r69dz\" (UID: \"8d9d6c34-100e-433a-a8cc-5a4628d08de6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz" Sep 09 23:24:49.195603 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.195561 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmqrl\" (UniqueName: \"kubernetes.io/projected/8d9d6c34-100e-433a-a8cc-5a4628d08de6-kube-api-access-pmqrl\") pod \"limitador-operator-controller-manager-74495c69f8-r69dz\" (UID: \"8d9d6c34-100e-433a-a8cc-5a4628d08de6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz" Sep 09 23:24:49.239452 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.239416 2580 generic.go:356] "Generic (PLEG): container finished" podID="6f6eae0f-cf42-4f44-a17d-d6526d7441a8" containerID="51fb4d8529fec1b9babb3b3a6204dccd691c469a40d8944180686de586e98ef6" exitCode=0 Sep 09 23:24:49.260938 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.260913 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" Sep 09 23:24:49.383859 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.383827 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz" Sep 09 23:24:49.387905 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.387885 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsb9x\" (UniqueName: \"kubernetes.io/projected/6f6eae0f-cf42-4f44-a17d-d6526d7441a8-kube-api-access-wsb9x\") pod \"6f6eae0f-cf42-4f44-a17d-d6526d7441a8\" (UID: \"6f6eae0f-cf42-4f44-a17d-d6526d7441a8\") " Sep 09 23:24:49.390110 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.390085 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f6eae0f-cf42-4f44-a17d-d6526d7441a8-kube-api-access-wsb9x" (OuterVolumeSpecName: "kube-api-access-wsb9x") pod "6f6eae0f-cf42-4f44-a17d-d6526d7441a8" (UID: "6f6eae0f-cf42-4f44-a17d-d6526d7441a8"). InnerVolumeSpecName "kube-api-access-wsb9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:24:49.488747 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.488716 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsb9x\" (UniqueName: \"kubernetes.io/projected/6f6eae0f-cf42-4f44-a17d-d6526d7441a8-kube-api-access-wsb9x\") on node \"ip-10-0-130-56.ec2.internal\" DevicePath \"\"" Sep 09 23:24:49.664600 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:24:49.664569 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d9d6c34_100e_433a_a8cc_5a4628d08de6.slice/crio-b2048ed4db570060770b772ed8538abfb5edbc959bfe8db8d7f2d1a6b3061bcc WatchSource:0}: Error finding container b2048ed4db570060770b772ed8538abfb5edbc959bfe8db8d7f2d1a6b3061bcc: Status 404 returned error can't find the container with id b2048ed4db570060770b772ed8538abfb5edbc959bfe8db8d7f2d1a6b3061bcc Sep 09 23:24:49.664801 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:49.664779 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz"] Sep 09 23:24:50.246366 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:50.246327 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz" event={"ID":"8d9d6c34-100e-433a-a8cc-5a4628d08de6","Type":"ContainerStarted","Data":"430324472dcce0ca7962a4a6c4461063e4552c2141de0f3b52330742c8033b9e"} Sep 09 23:24:50.246782 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:50.246386 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz" event={"ID":"8d9d6c34-100e-433a-a8cc-5a4628d08de6","Type":"ContainerStarted","Data":"b2048ed4db570060770b772ed8538abfb5edbc959bfe8db8d7f2d1a6b3061bcc"} Sep 09 23:24:50.246782 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:50.246505 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz" Sep 09 23:24:50.247656 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:50.247642 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ff4hs" Sep 09 23:24:50.247715 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:50.247666 2580 scope.go:117] "RemoveContainer" containerID="51fb4d8529fec1b9babb3b3a6204dccd691c469a40d8944180686de586e98ef6" Sep 09 23:24:50.269409 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:50.269368 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz" podStartSLOduration=1.269352954 podStartE2EDuration="1.269352954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:24:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:24:50.26859628 +0000 UTC m=+451.695019588" watchObservedRunningTime="2026-09-09 23:24:50.269352954 +0000 UTC m=+451.695776262" Sep 09 23:24:51.123975 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:24:51.123934 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f6eae0f-cf42-4f44-a17d-d6526d7441a8" path="/var/lib/kubelet/pods/6f6eae0f-cf42-4f44-a17d-d6526d7441a8/volumes" Sep 09 23:25:01.256127 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:01.256095 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dz" Sep 09 23:25:17.228162 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.228126 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq"] Sep 09 23:25:17.231338 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.231318 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.233806 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.233784 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mn8zs\"" Sep 09 23:25:17.243947 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.243924 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq"] Sep 09 23:25:17.326369 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.326331 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.326544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.326388 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f5a84b3-8aed-44ba-ab78-ddae43293248-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.326544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.326446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tpd8\" (UniqueName: \"kubernetes.io/projected/3f5a84b3-8aed-44ba-ab78-ddae43293248-kube-api-access-6tpd8\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.326544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.326475 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f5a84b3-8aed-44ba-ab78-ddae43293248-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.326544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.326508 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f5a84b3-8aed-44ba-ab78-ddae43293248-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.326544 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.326533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.326751 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.326607 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.326751 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.326657 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.326751 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.326676 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.427872 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.427837 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428049 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.427883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f5a84b3-8aed-44ba-ab78-ddae43293248-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428049 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.428010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpd8\" (UniqueName: \"kubernetes.io/projected/3f5a84b3-8aed-44ba-ab78-ddae43293248-kube-api-access-6tpd8\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428141 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.428056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f5a84b3-8aed-44ba-ab78-ddae43293248-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428141 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.428095 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f5a84b3-8aed-44ba-ab78-ddae43293248-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428141 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.428126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428290 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.428171 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428290 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.428205 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428290 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.428246 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428723 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.428307 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428723 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.428517 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428723 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.428569 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428723 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.428587 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.428936 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.428876 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f5a84b3-8aed-44ba-ab78-ddae43293248-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.430429 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.430408 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f5a84b3-8aed-44ba-ab78-ddae43293248-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.430646 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.430626 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f5a84b3-8aed-44ba-ab78-ddae43293248-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.436308 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.436284 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f5a84b3-8aed-44ba-ab78-ddae43293248-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.436466 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.436447 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tpd8\" (UniqueName: \"kubernetes.io/projected/3f5a84b3-8aed-44ba-ab78-ddae43293248-kube-api-access-6tpd8\") pod \"maas-default-gateway-openshift-default-78bb78ff78-rv5wq\" (UID: \"3f5a84b3-8aed-44ba-ab78-ddae43293248\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.544413 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.544375 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:17.831839 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.831813 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq"] Sep 09 23:25:17.834014 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:25:17.833985 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5a84b3_8aed_44ba_ab78_ddae43293248.slice/crio-542ae78a83580d5d2fa838cf4adc1ab548f8a9cc2cacb1ad4095020c69542af8 WatchSource:0}: Error finding container 542ae78a83580d5d2fa838cf4adc1ab548f8a9cc2cacb1ad4095020c69542af8: Status 404 returned error can't find the container with id 542ae78a83580d5d2fa838cf4adc1ab548f8a9cc2cacb1ad4095020c69542af8 Sep 09 23:25:17.836381 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.836346 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 09 23:25:17.836480 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.836413 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 09 23:25:17.836480 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:17.836442 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 09 23:25:18.406084 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:18.406051 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" event={"ID":"3f5a84b3-8aed-44ba-ab78-ddae43293248","Type":"ContainerStarted","Data":"2cde61d5f3931c34711c79a42f32182fa1a68e910c7b1f4b03ca3be83390cd15"} Sep 09 23:25:18.406495 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:18.406100 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" event={"ID":"3f5a84b3-8aed-44ba-ab78-ddae43293248","Type":"ContainerStarted","Data":"542ae78a83580d5d2fa838cf4adc1ab548f8a9cc2cacb1ad4095020c69542af8"} Sep 09 23:25:18.426482 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:18.426439 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" podStartSLOduration=1.426424956 podStartE2EDuration="1.426424956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:25:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:25:18.424666397 +0000 UTC m=+479.851089703" watchObservedRunningTime="2026-09-09 23:25:18.426424956 +0000 UTC m=+479.852848329" Sep 09 23:25:18.545620 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:18.545582 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:18.550115 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:18.550092 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:19.411620 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.411533 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:19.412430 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.412411 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:25:19.694818 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.694737 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tpxwn"] Sep 09 23:25:19.699388 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.699369 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" Sep 09 23:25:19.701814 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.701787 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7nt94\"" Sep 09 23:25:19.701937 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.701847 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 09 23:25:19.707607 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.707228 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tpxwn"] Sep 09 23:25:19.738219 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.738190 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tpxwn"] Sep 09 23:25:19.747297 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.747269 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkg6l\" (UniqueName: \"kubernetes.io/projected/1f55521b-1fe7-4069-b578-ec8b253486a5-kube-api-access-hkg6l\") pod \"limitador-limitador-84994f7ffc-tpxwn\" (UID: \"1f55521b-1fe7-4069-b578-ec8b253486a5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" Sep 09 23:25:19.747402 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.747315 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1f55521b-1fe7-4069-b578-ec8b253486a5-config-file\") pod \"limitador-limitador-84994f7ffc-tpxwn\" (UID: \"1f55521b-1fe7-4069-b578-ec8b253486a5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" Sep 09 23:25:19.848227 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.848193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkg6l\" (UniqueName: \"kubernetes.io/projected/1f55521b-1fe7-4069-b578-ec8b253486a5-kube-api-access-hkg6l\") pod \"limitador-limitador-84994f7ffc-tpxwn\" (UID: \"1f55521b-1fe7-4069-b578-ec8b253486a5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" Sep 09 23:25:19.848389 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.848251 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1f55521b-1fe7-4069-b578-ec8b253486a5-config-file\") pod \"limitador-limitador-84994f7ffc-tpxwn\" (UID: \"1f55521b-1fe7-4069-b578-ec8b253486a5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" Sep 09 23:25:19.848951 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.848924 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1f55521b-1fe7-4069-b578-ec8b253486a5-config-file\") pod \"limitador-limitador-84994f7ffc-tpxwn\" (UID: \"1f55521b-1fe7-4069-b578-ec8b253486a5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" Sep 09 23:25:19.856084 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:19.856059 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkg6l\" (UniqueName: \"kubernetes.io/projected/1f55521b-1fe7-4069-b578-ec8b253486a5-kube-api-access-hkg6l\") pod \"limitador-limitador-84994f7ffc-tpxwn\" (UID: \"1f55521b-1fe7-4069-b578-ec8b253486a5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" Sep 09 23:25:20.011281 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:20.011196 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" Sep 09 23:25:20.279197 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:20.279160 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tpxwn"] Sep 09 23:25:20.280659 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:25:20.280631 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f55521b_1fe7_4069_b578_ec8b253486a5.slice/crio-a006857c0429c174557bef50a30675f3ee918cc5c595be56e5db983883071d28 WatchSource:0}: Error finding container a006857c0429c174557bef50a30675f3ee918cc5c595be56e5db983883071d28: Status 404 returned error can't find the container with id a006857c0429c174557bef50a30675f3ee918cc5c595be56e5db983883071d28 Sep 09 23:25:20.418378 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:20.418345 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" event={"ID":"1f55521b-1fe7-4069-b578-ec8b253486a5","Type":"ContainerStarted","Data":"a006857c0429c174557bef50a30675f3ee918cc5c595be56e5db983883071d28"} Sep 09 23:25:21.425419 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:21.425334 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" event={"ID":"1f55521b-1fe7-4069-b578-ec8b253486a5","Type":"ContainerStarted","Data":"8978d292e667d22d2638d6fe2a891b46c849b956dc8d0cde3c4abae428c911b7"} Sep 09 23:25:21.425780 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:21.425618 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" Sep 09 23:25:21.443061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:21.443016 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" podStartSLOduration=1.5795269159999998 podStartE2EDuration="2.443000808s" totalImagesPullingTime="863.473892ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:25:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:25:21.440921743 +0000 UTC m=+482.867345050" watchObservedRunningTime="2026-09-09 23:25:21.443000808 +0000 UTC m=+482.869424115" Sep 09 23:25:32.431444 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:25:32.431411 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-tpxwn" Sep 09 23:26:32.524452 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:32.524356 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tpxwn"] Sep 09 23:26:41.868139 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:41.868063 2580 generic.go:356] "Generic (PLEG): container finished" podID="3f5a84b3-8aed-44ba-ab78-ddae43293248" containerID="2cde61d5f3931c34711c79a42f32182fa1a68e910c7b1f4b03ca3be83390cd15" exitCode=0 Sep 09 23:26:41.868139 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:41.868115 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" event={"ID":"3f5a84b3-8aed-44ba-ab78-ddae43293248","Type":"ContainerDied","Data":"2cde61d5f3931c34711c79a42f32182fa1a68e910c7b1f4b03ca3be83390cd15"} Sep 09 23:26:41.868561 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:41.868497 2580 scope.go:117] "RemoveContainer" containerID="2cde61d5f3931c34711c79a42f32182fa1a68e910c7b1f4b03ca3be83390cd15" Sep 09 23:26:41.869510 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:41.869481 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 09 23:26:41.869579 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:41.869546 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 09 23:26:41.869617 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:41.869582 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 09 23:26:42.875452 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:42.875415 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" event={"ID":"3f5a84b3-8aed-44ba-ab78-ddae43293248","Type":"ContainerStarted","Data":"bacc7de8450e02e8bd9e75fbd700c198d6304635f355fdb8ea769448462ace50"} Sep 09 23:26:43.545074 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:43.545036 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:26:43.545329 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:43.545291 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-rv5wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 09 23:26:43.545389 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:43.545366 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" podUID="3f5a84b3-8aed-44ba-ab78-ddae43293248" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 09 23:26:44.544866 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:44.544831 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-rv5wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 09 23:26:44.545254 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:44.544905 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" podUID="3f5a84b3-8aed-44ba-ab78-ddae43293248" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 09 23:26:45.545873 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:45.545834 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-rv5wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 09 23:26:45.546287 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:45.545914 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" podUID="3f5a84b3-8aed-44ba-ab78-ddae43293248" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 09 23:26:46.545599 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:46.545561 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-rv5wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 09 23:26:46.545778 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:46.545616 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" podUID="3f5a84b3-8aed-44ba-ab78-ddae43293248" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 09 23:26:47.545098 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:47.545064 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-rv5wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 09 23:26:47.545567 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:47.545145 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" podUID="3f5a84b3-8aed-44ba-ab78-ddae43293248" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 09 23:26:48.549183 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:48.549146 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:26:48.549641 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:48.549533 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:26:48.550158 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:48.550136 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rv5wq" Sep 09 23:26:49.911253 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:49.911218 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tpxwn"] Sep 09 23:26:57.286864 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.286824 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj"] Sep 09 23:26:57.289812 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.289788 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.293303 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.293273 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 09 23:26:57.293303 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.293272 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pq5fv\"" Sep 09 23:26:57.293494 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.293335 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 09 23:26:57.293494 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.293273 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 09 23:26:57.297823 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.297800 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj"] Sep 09 23:26:57.390215 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.390166 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26z45\" (UniqueName: \"kubernetes.io/projected/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-kube-api-access-26z45\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.390215 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.390227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.390458 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.390264 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.390458 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.390292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.390458 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.390313 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.390458 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.390356 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.491028 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.490955 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.491219 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.491048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.491219 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.491079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.491319 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.491237 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.491376 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.491332 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26z45\" (UniqueName: \"kubernetes.io/projected/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-kube-api-access-26z45\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.491376 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.491365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.491486 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.491387 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.491486 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.491440 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.491654 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.491633 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.493356 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.493338 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.493628 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.493611 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.502284 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.502264 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26z45\" (UniqueName: \"kubernetes.io/projected/b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4-kube-api-access-26z45\") pod \"e2e-distinct-simulated-kserve-6685d86975-hkzrj\" (UID: \"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.601624 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.601529 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:26:57.876021 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.875995 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj"] Sep 09 23:26:57.878188 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:26:57.878148 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4bc94ae_a9bb_439f_8fd3_8e64cc5a83d4.slice/crio-1905eea35b966edd9a4706db03b0b32d240238b3ab9ccbeae2f89a527c652bc8 WatchSource:0}: Error finding container 1905eea35b966edd9a4706db03b0b32d240238b3ab9ccbeae2f89a527c652bc8: Status 404 returned error can't find the container with id 1905eea35b966edd9a4706db03b0b32d240238b3ab9ccbeae2f89a527c652bc8 Sep 09 23:26:57.895949 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.895910 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tpxwn"] Sep 09 23:26:57.956796 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:57.956768 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" event={"ID":"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4","Type":"ContainerStarted","Data":"1905eea35b966edd9a4706db03b0b32d240238b3ab9ccbeae2f89a527c652bc8"} Sep 09 23:26:58.963444 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:58.963412 2580 generic.go:356] "Generic (PLEG): container finished" podID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" containerID="579132232c2d54a2758453894c61093cd9277a125a99e20e307c59e6d3399b00" exitCode=0 Sep 09 23:26:58.963821 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:58.963497 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" event={"ID":"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4","Type":"ContainerDied","Data":"579132232c2d54a2758453894c61093cd9277a125a99e20e307c59e6d3399b00"} Sep 09 23:26:58.963821 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:58.963711 2580 scope.go:117] "RemoveContainer" containerID="579132232c2d54a2758453894c61093cd9277a125a99e20e307c59e6d3399b00" Sep 09 23:26:59.976380 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:59.976345 2580 generic.go:356] "Generic (PLEG): container finished" podID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" containerID="312705619b23ae7f6e05e6a1ab6fb347f638a3fa436dd3e408f064d564ed6d14" exitCode=0 Sep 09 23:26:59.976784 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:59.976431 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" event={"ID":"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4","Type":"ContainerDied","Data":"312705619b23ae7f6e05e6a1ab6fb347f638a3fa436dd3e408f064d564ed6d14"} Sep 09 23:26:59.976784 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:59.976517 2580 scope.go:117] "RemoveContainer" containerID="579132232c2d54a2758453894c61093cd9277a125a99e20e307c59e6d3399b00" Sep 09 23:26:59.976784 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:26:59.976749 2580 scope.go:117] "RemoveContainer" containerID="312705619b23ae7f6e05e6a1ab6fb347f638a3fa436dd3e408f064d564ed6d14" Sep 09 23:26:59.977037 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:26:59.977012 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:27:00.983559 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:00.983534 2580 scope.go:117] "RemoveContainer" containerID="312705619b23ae7f6e05e6a1ab6fb347f638a3fa436dd3e408f064d564ed6d14" Sep 09 23:27:00.983997 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:27:00.983729 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:27:07.602644 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:07.602610 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:27:07.603067 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:07.602702 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:27:07.603067 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:07.602743 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:27:07.603067 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:07.602759 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:27:07.603178 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:07.603088 2580 scope.go:117] "RemoveContainer" containerID="312705619b23ae7f6e05e6a1ab6fb347f638a3fa436dd3e408f064d564ed6d14" Sep 09 23:27:07.603279 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:27:07.603260 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:27:08.021215 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:08.021134 2580 scope.go:117] "RemoveContainer" containerID="312705619b23ae7f6e05e6a1ab6fb347f638a3fa436dd3e408f064d564ed6d14" Sep 09 23:27:08.021349 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:27:08.021316 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:27:13.297210 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:13.297176 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tpxwn"] Sep 09 23:27:19.135939 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:19.135912 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/0.log" Sep 09 23:27:19.136505 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:19.136485 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/0.log" Sep 09 23:27:20.201088 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:20.201052 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tpxwn"] Sep 09 23:27:22.118226 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:22.118198 2580 scope.go:117] "RemoveContainer" containerID="312705619b23ae7f6e05e6a1ab6fb347f638a3fa436dd3e408f064d564ed6d14" Sep 09 23:27:23.103650 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:23.103618 2580 generic.go:356] "Generic (PLEG): container finished" podID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" containerID="ef259127a4933d2a2d2fe6152afc329642ecdcc400a5a8c691ba2284401d2993" exitCode=0 Sep 09 23:27:23.103812 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:23.103673 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" event={"ID":"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4","Type":"ContainerDied","Data":"ef259127a4933d2a2d2fe6152afc329642ecdcc400a5a8c691ba2284401d2993"} Sep 09 23:27:23.103812 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:23.103720 2580 scope.go:117] "RemoveContainer" containerID="312705619b23ae7f6e05e6a1ab6fb347f638a3fa436dd3e408f064d564ed6d14" Sep 09 23:27:23.104123 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:23.104104 2580 scope.go:117] "RemoveContainer" containerID="ef259127a4933d2a2d2fe6152afc329642ecdcc400a5a8c691ba2284401d2993" Sep 09 23:27:23.104312 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:27:23.104288 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:27:24.102327 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.102296 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667"] Sep 09 23:27:24.105663 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.105643 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.108140 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.108106 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 09 23:27:24.113455 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.113430 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667"] Sep 09 23:27:24.128537 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.128512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab05e351-2fbe-48ff-85fa-d34c892a86df-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.128653 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.128542 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab05e351-2fbe-48ff-85fa-d34c892a86df-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.128653 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.128567 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7fv2\" (UniqueName: \"kubernetes.io/projected/ab05e351-2fbe-48ff-85fa-d34c892a86df-kube-api-access-l7fv2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.128745 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.128702 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab05e351-2fbe-48ff-85fa-d34c892a86df-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.128788 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.128742 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab05e351-2fbe-48ff-85fa-d34c892a86df-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.128827 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.128816 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab05e351-2fbe-48ff-85fa-d34c892a86df-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.230239 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.230206 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab05e351-2fbe-48ff-85fa-d34c892a86df-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.230239 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.230243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab05e351-2fbe-48ff-85fa-d34c892a86df-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.230461 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.230285 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab05e351-2fbe-48ff-85fa-d34c892a86df-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.230461 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.230322 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab05e351-2fbe-48ff-85fa-d34c892a86df-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.230461 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.230339 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab05e351-2fbe-48ff-85fa-d34c892a86df-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.230461 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.230375 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7fv2\" (UniqueName: \"kubernetes.io/projected/ab05e351-2fbe-48ff-85fa-d34c892a86df-kube-api-access-l7fv2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.230723 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.230701 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab05e351-2fbe-48ff-85fa-d34c892a86df-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.230794 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.230762 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab05e351-2fbe-48ff-85fa-d34c892a86df-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.230794 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.230778 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab05e351-2fbe-48ff-85fa-d34c892a86df-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.232514 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.232488 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab05e351-2fbe-48ff-85fa-d34c892a86df-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.232686 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.232668 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab05e351-2fbe-48ff-85fa-d34c892a86df-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.237329 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.237308 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7fv2\" (UniqueName: \"kubernetes.io/projected/ab05e351-2fbe-48ff-85fa-d34c892a86df-kube-api-access-l7fv2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667\" (UID: \"ab05e351-2fbe-48ff-85fa-d34c892a86df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.417517 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.417434 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:24.685870 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.685843 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667"] Sep 09 23:27:24.686910 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:27:24.686887 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab05e351_2fbe_48ff_85fa_d34c892a86df.slice/crio-a2a8d477134e13c09800b313e5e5184d74f6a88bcdb22d121e8769bd4f2f8c13 WatchSource:0}: Error finding container a2a8d477134e13c09800b313e5e5184d74f6a88bcdb22d121e8769bd4f2f8c13: Status 404 returned error can't find the container with id a2a8d477134e13c09800b313e5e5184d74f6a88bcdb22d121e8769bd4f2f8c13 Sep 09 23:27:24.688336 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:24.688315 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 23:27:25.117622 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:25.117591 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" event={"ID":"ab05e351-2fbe-48ff-85fa-d34c892a86df","Type":"ContainerStarted","Data":"59d4ed366e47dc94e04ce76144b5830cce8809ca1b0dc6615d7d0d2f42c12a4b"} Sep 09 23:27:25.118145 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:25.117641 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" event={"ID":"ab05e351-2fbe-48ff-85fa-d34c892a86df","Type":"ContainerStarted","Data":"a2a8d477134e13c09800b313e5e5184d74f6a88bcdb22d121e8769bd4f2f8c13"} Sep 09 23:27:25.135437 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:25.135382 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" podStartSLOduration=0.982433614 podStartE2EDuration="1.135366718s" totalImagesPullingTime="152.933104ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:27:25.134250873 +0000 UTC m=+606.560674182" watchObservedRunningTime="2026-09-09 23:27:25.135366718 +0000 UTC m=+606.561790077" Sep 09 23:27:25.502806 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:25.502730 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tpxwn"] Sep 09 23:27:27.601809 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:27.601774 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:27:27.602218 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:27.601834 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:27:27.602218 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:27.601850 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:27:27.602218 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:27.601862 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:27:27.602329 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:27.602235 2580 scope.go:117] "RemoveContainer" containerID="ef259127a4933d2a2d2fe6152afc329642ecdcc400a5a8c691ba2284401d2993" Sep 09 23:27:27.602419 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:27:27.602399 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:27:34.418328 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:34.418282 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:34.418786 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:34.418376 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:34.430775 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:34.430752 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:35.182669 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:35.182633 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667" Sep 09 23:27:38.294115 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:38.294077 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tpxwn"] Sep 09 23:27:41.118846 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:41.118769 2580 scope.go:117] "RemoveContainer" containerID="ef259127a4933d2a2d2fe6152afc329642ecdcc400a5a8c691ba2284401d2993" Sep 09 23:27:41.119239 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:27:41.118933 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:27:44.488623 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.488590 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7"] Sep 09 23:27:44.492210 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.492187 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.494529 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.494499 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Sep 09 23:27:44.504867 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.504835 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7"] Sep 09 23:27:44.616076 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.616015 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76b303ca-932f-4459-bd7d-3c1e70189b5e-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.616076 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.616063 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76b303ca-932f-4459-bd7d-3c1e70189b5e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.616451 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.616109 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76b303ca-932f-4459-bd7d-3c1e70189b5e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.616451 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.616152 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6pmr\" (UniqueName: \"kubernetes.io/projected/76b303ca-932f-4459-bd7d-3c1e70189b5e-kube-api-access-m6pmr\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.616451 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.616254 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76b303ca-932f-4459-bd7d-3c1e70189b5e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.616451 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.616310 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76b303ca-932f-4459-bd7d-3c1e70189b5e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.717644 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.717594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6pmr\" (UniqueName: \"kubernetes.io/projected/76b303ca-932f-4459-bd7d-3c1e70189b5e-kube-api-access-m6pmr\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.717813 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.717686 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76b303ca-932f-4459-bd7d-3c1e70189b5e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.717813 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.717721 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76b303ca-932f-4459-bd7d-3c1e70189b5e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.717813 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.717742 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76b303ca-932f-4459-bd7d-3c1e70189b5e-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.717813 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.717757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76b303ca-932f-4459-bd7d-3c1e70189b5e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.717813 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.717785 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76b303ca-932f-4459-bd7d-3c1e70189b5e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.718183 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.718158 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76b303ca-932f-4459-bd7d-3c1e70189b5e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.718183 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.718175 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76b303ca-932f-4459-bd7d-3c1e70189b5e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.718307 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.718219 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76b303ca-932f-4459-bd7d-3c1e70189b5e-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.720056 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.720028 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76b303ca-932f-4459-bd7d-3c1e70189b5e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.720286 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.720269 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76b303ca-932f-4459-bd7d-3c1e70189b5e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.725245 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.725220 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6pmr\" (UniqueName: \"kubernetes.io/projected/76b303ca-932f-4459-bd7d-3c1e70189b5e-kube-api-access-m6pmr\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7\" (UID: \"76b303ca-932f-4459-bd7d-3c1e70189b5e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:44.804317 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:44.804292 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:45.073658 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:45.073628 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7"] Sep 09 23:27:45.074686 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:27:45.074661 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76b303ca_932f_4459_bd7d_3c1e70189b5e.slice/crio-4cc9b0901c6c244e4341e78592f842aa217ec0b1140e30e6ff960da347445c76 WatchSource:0}: Error finding container 4cc9b0901c6c244e4341e78592f842aa217ec0b1140e30e6ff960da347445c76: Status 404 returned error can't find the container with id 4cc9b0901c6c244e4341e78592f842aa217ec0b1140e30e6ff960da347445c76 Sep 09 23:27:45.226952 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:45.226916 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" event={"ID":"76b303ca-932f-4459-bd7d-3c1e70189b5e","Type":"ContainerStarted","Data":"4cc9b0901c6c244e4341e78592f842aa217ec0b1140e30e6ff960da347445c76"} Sep 09 23:27:46.233799 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:46.233764 2580 generic.go:356] "Generic (PLEG): container finished" podID="76b303ca-932f-4459-bd7d-3c1e70189b5e" containerID="88dac1899c5ba42b8e72fe1df20e7e570f8df25aafb4198d6b4e45bb51c43255" exitCode=0 Sep 09 23:27:46.234261 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:46.233834 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" event={"ID":"76b303ca-932f-4459-bd7d-3c1e70189b5e","Type":"ContainerDied","Data":"88dac1899c5ba42b8e72fe1df20e7e570f8df25aafb4198d6b4e45bb51c43255"} Sep 09 23:27:46.234261 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:46.234071 2580 scope.go:117] "RemoveContainer" containerID="88dac1899c5ba42b8e72fe1df20e7e570f8df25aafb4198d6b4e45bb51c43255" Sep 09 23:27:46.602495 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:46.602468 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tpxwn"] Sep 09 23:27:47.242186 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:47.242151 2580 generic.go:356] "Generic (PLEG): container finished" podID="76b303ca-932f-4459-bd7d-3c1e70189b5e" containerID="66d3fd4d28065f91b6a97e32fd3fc2d68826f002a082ec45a08a69ddc2376e2e" exitCode=0 Sep 09 23:27:47.242633 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:47.242232 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" event={"ID":"76b303ca-932f-4459-bd7d-3c1e70189b5e","Type":"ContainerDied","Data":"66d3fd4d28065f91b6a97e32fd3fc2d68826f002a082ec45a08a69ddc2376e2e"} Sep 09 23:27:47.242633 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:47.242287 2580 scope.go:117] "RemoveContainer" containerID="88dac1899c5ba42b8e72fe1df20e7e570f8df25aafb4198d6b4e45bb51c43255" Sep 09 23:27:47.242633 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:47.242465 2580 scope.go:117] "RemoveContainer" containerID="66d3fd4d28065f91b6a97e32fd3fc2d68826f002a082ec45a08a69ddc2376e2e" Sep 09 23:27:47.242757 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:27:47.242670 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:27:48.250926 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:48.250890 2580 scope.go:117] "RemoveContainer" containerID="66d3fd4d28065f91b6a97e32fd3fc2d68826f002a082ec45a08a69ddc2376e2e" Sep 09 23:27:48.251333 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:27:48.251136 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:27:52.119113 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:52.119081 2580 scope.go:117] "RemoveContainer" containerID="ef259127a4933d2a2d2fe6152afc329642ecdcc400a5a8c691ba2284401d2993" Sep 09 23:27:53.276407 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:53.276374 2580 generic.go:356] "Generic (PLEG): container finished" podID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" containerID="594706d6adeca228b73fc4a870e027b0a82f1565e74f06db2089f2b91a15fdc4" exitCode=0 Sep 09 23:27:53.276821 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:53.276447 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" event={"ID":"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4","Type":"ContainerDied","Data":"594706d6adeca228b73fc4a870e027b0a82f1565e74f06db2089f2b91a15fdc4"} Sep 09 23:27:53.276821 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:53.276524 2580 scope.go:117] "RemoveContainer" containerID="ef259127a4933d2a2d2fe6152afc329642ecdcc400a5a8c691ba2284401d2993" Sep 09 23:27:53.276952 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:53.276867 2580 scope.go:117] "RemoveContainer" containerID="594706d6adeca228b73fc4a870e027b0a82f1565e74f06db2089f2b91a15fdc4" Sep 09 23:27:53.277076 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:27:53.277058 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:27:54.805240 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:54.805205 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:54.805609 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:54.805262 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:54.805609 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:54.805274 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:54.805609 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:54.805283 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:27:54.805715 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:54.805625 2580 scope.go:117] "RemoveContainer" containerID="66d3fd4d28065f91b6a97e32fd3fc2d68826f002a082ec45a08a69ddc2376e2e" Sep 09 23:27:54.805792 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:27:54.805774 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:27:57.602605 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:57.602561 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:27:57.603061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:57.602628 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:27:57.603061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:57.602649 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:27:57.603061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:57.602664 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:27:57.603061 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:27:57.603045 2580 scope.go:117] "RemoveContainer" containerID="594706d6adeca228b73fc4a870e027b0a82f1565e74f06db2089f2b91a15fdc4" Sep 09 23:27:57.603263 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:27:57.603228 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:28:06.119100 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:06.119062 2580 scope.go:117] "RemoveContainer" containerID="66d3fd4d28065f91b6a97e32fd3fc2d68826f002a082ec45a08a69ddc2376e2e" Sep 09 23:28:07.354227 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:07.354183 2580 generic.go:356] "Generic (PLEG): container finished" podID="76b303ca-932f-4459-bd7d-3c1e70189b5e" containerID="89b853877c3a25bf9c2c1dc5e8015bb70c1ce936599c6d23d664d51df0c4e6d6" exitCode=0 Sep 09 23:28:07.354669 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:07.354244 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" event={"ID":"76b303ca-932f-4459-bd7d-3c1e70189b5e","Type":"ContainerDied","Data":"89b853877c3a25bf9c2c1dc5e8015bb70c1ce936599c6d23d664d51df0c4e6d6"} Sep 09 23:28:07.354669 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:07.354289 2580 scope.go:117] "RemoveContainer" containerID="66d3fd4d28065f91b6a97e32fd3fc2d68826f002a082ec45a08a69ddc2376e2e" Sep 09 23:28:07.354669 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:07.354632 2580 scope.go:117] "RemoveContainer" containerID="89b853877c3a25bf9c2c1dc5e8015bb70c1ce936599c6d23d664d51df0c4e6d6" Sep 09 23:28:07.354868 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:28:07.354845 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:28:11.118387 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:11.118354 2580 scope.go:117] "RemoveContainer" containerID="594706d6adeca228b73fc4a870e027b0a82f1565e74f06db2089f2b91a15fdc4" Sep 09 23:28:11.118796 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:28:11.118541 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:28:14.805259 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:14.805223 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:28:14.805259 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:14.805269 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:28:14.805914 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:14.805280 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:28:14.805914 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:14.805290 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:28:14.805914 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:14.805695 2580 scope.go:117] "RemoveContainer" containerID="89b853877c3a25bf9c2c1dc5e8015bb70c1ce936599c6d23d664d51df0c4e6d6" Sep 09 23:28:14.806112 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:28:14.805912 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:28:15.398508 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:15.398480 2580 scope.go:117] "RemoveContainer" containerID="89b853877c3a25bf9c2c1dc5e8015bb70c1ce936599c6d23d664d51df0c4e6d6" Sep 09 23:28:15.398684 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:28:15.398644 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:28:26.119039 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:26.119003 2580 scope.go:117] "RemoveContainer" containerID="594706d6adeca228b73fc4a870e027b0a82f1565e74f06db2089f2b91a15fdc4" Sep 09 23:28:26.119508 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:28:26.119167 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:28:27.119132 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:27.119100 2580 scope.go:117] "RemoveContainer" containerID="89b853877c3a25bf9c2c1dc5e8015bb70c1ce936599c6d23d664d51df0c4e6d6" Sep 09 23:28:27.461889 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:27.461859 2580 generic.go:356] "Generic (PLEG): container finished" podID="76b303ca-932f-4459-bd7d-3c1e70189b5e" containerID="c4669b10fd57adf5cf164928fe676edb392caa64ca5b779697424a2f56b5d11a" exitCode=0 Sep 09 23:28:27.462055 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:27.461938 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" event={"ID":"76b303ca-932f-4459-bd7d-3c1e70189b5e","Type":"ContainerDied","Data":"c4669b10fd57adf5cf164928fe676edb392caa64ca5b779697424a2f56b5d11a"} Sep 09 23:28:27.462055 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:27.462016 2580 scope.go:117] "RemoveContainer" containerID="89b853877c3a25bf9c2c1dc5e8015bb70c1ce936599c6d23d664d51df0c4e6d6" Sep 09 23:28:27.462385 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:27.462362 2580 scope.go:117] "RemoveContainer" containerID="c4669b10fd57adf5cf164928fe676edb392caa64ca5b779697424a2f56b5d11a" Sep 09 23:28:27.462591 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:28:27.462569 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:28:34.804913 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:34.804873 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:28:34.805392 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:34.804931 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:28:34.805392 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:34.804945 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:28:34.805392 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:34.804954 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:28:34.805392 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:34.805349 2580 scope.go:117] "RemoveContainer" containerID="c4669b10fd57adf5cf164928fe676edb392caa64ca5b779697424a2f56b5d11a" Sep 09 23:28:34.805547 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:28:34.805524 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:28:41.118404 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:41.118370 2580 scope.go:117] "RemoveContainer" containerID="594706d6adeca228b73fc4a870e027b0a82f1565e74f06db2089f2b91a15fdc4" Sep 09 23:28:41.538019 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:41.537981 2580 generic.go:356] "Generic (PLEG): container finished" podID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" containerID="55e737020be1c68908903f671bebbd067a5fe574dfd88d5a2a3eb8b9878e58cb" exitCode=0 Sep 09 23:28:41.538019 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:41.537995 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" event={"ID":"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4","Type":"ContainerDied","Data":"55e737020be1c68908903f671bebbd067a5fe574dfd88d5a2a3eb8b9878e58cb"} Sep 09 23:28:41.538261 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:41.538081 2580 scope.go:117] "RemoveContainer" containerID="594706d6adeca228b73fc4a870e027b0a82f1565e74f06db2089f2b91a15fdc4" Sep 09 23:28:41.538419 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:41.538400 2580 scope.go:117] "RemoveContainer" containerID="55e737020be1c68908903f671bebbd067a5fe574dfd88d5a2a3eb8b9878e58cb" Sep 09 23:28:41.538596 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:28:41.538579 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:28:47.601699 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:47.601661 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:28:47.602222 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:47.601746 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:28:47.602222 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:47.601763 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:28:47.602222 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:47.601776 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:28:47.602222 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:47.602122 2580 scope.go:117] "RemoveContainer" containerID="55e737020be1c68908903f671bebbd067a5fe574dfd88d5a2a3eb8b9878e58cb" Sep 09 23:28:47.602470 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:28:47.602296 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:28:50.118575 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:28:50.118541 2580 scope.go:117] "RemoveContainer" containerID="c4669b10fd57adf5cf164928fe676edb392caa64ca5b779697424a2f56b5d11a" Sep 09 23:28:50.118958 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:28:50.118715 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:29:01.118454 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:01.118410 2580 scope.go:117] "RemoveContainer" containerID="55e737020be1c68908903f671bebbd067a5fe574dfd88d5a2a3eb8b9878e58cb" Sep 09 23:29:01.118898 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:29:01.118662 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:29:02.118594 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:02.118558 2580 scope.go:117] "RemoveContainer" containerID="c4669b10fd57adf5cf164928fe676edb392caa64ca5b779697424a2f56b5d11a" Sep 09 23:29:02.119027 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:29:02.118712 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:29:14.118524 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:14.118437 2580 scope.go:117] "RemoveContainer" containerID="55e737020be1c68908903f671bebbd067a5fe574dfd88d5a2a3eb8b9878e58cb" Sep 09 23:29:14.119095 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:29:14.118688 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:29:16.118569 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:16.118536 2580 scope.go:117] "RemoveContainer" containerID="c4669b10fd57adf5cf164928fe676edb392caa64ca5b779697424a2f56b5d11a" Sep 09 23:29:16.734507 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:16.734425 2580 generic.go:356] "Generic (PLEG): container finished" podID="76b303ca-932f-4459-bd7d-3c1e70189b5e" containerID="39ec76b0ad57a2d67853eaa4b682ba87609a6ef80562304999ccada56185f4e2" exitCode=0 Sep 09 23:29:16.734507 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:16.734469 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" event={"ID":"76b303ca-932f-4459-bd7d-3c1e70189b5e","Type":"ContainerDied","Data":"39ec76b0ad57a2d67853eaa4b682ba87609a6ef80562304999ccada56185f4e2"} Sep 09 23:29:16.734507 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:16.734505 2580 scope.go:117] "RemoveContainer" containerID="c4669b10fd57adf5cf164928fe676edb392caa64ca5b779697424a2f56b5d11a" Sep 09 23:29:16.734860 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:16.734840 2580 scope.go:117] "RemoveContainer" containerID="39ec76b0ad57a2d67853eaa4b682ba87609a6ef80562304999ccada56185f4e2" Sep 09 23:29:16.735095 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:29:16.735072 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:29:24.805075 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:24.805024 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:29:24.805075 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:24.805080 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:29:24.805762 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:24.805094 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:29:24.805762 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:24.805106 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:29:24.805762 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:24.805579 2580 scope.go:117] "RemoveContainer" containerID="39ec76b0ad57a2d67853eaa4b682ba87609a6ef80562304999ccada56185f4e2" Sep 09 23:29:24.805917 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:29:24.805806 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:29:29.123166 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:29.123133 2580 scope.go:117] "RemoveContainer" containerID="55e737020be1c68908903f671bebbd067a5fe574dfd88d5a2a3eb8b9878e58cb" Sep 09 23:29:29.123571 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:29:29.123330 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:29:37.119174 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:37.119132 2580 scope.go:117] "RemoveContainer" containerID="39ec76b0ad57a2d67853eaa4b682ba87609a6ef80562304999ccada56185f4e2" Sep 09 23:29:37.119633 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:29:37.119386 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:29:41.119059 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:41.119022 2580 scope.go:117] "RemoveContainer" containerID="55e737020be1c68908903f671bebbd067a5fe574dfd88d5a2a3eb8b9878e58cb" Sep 09 23:29:41.119502 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:29:41.119268 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:29:48.118708 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:48.118673 2580 scope.go:117] "RemoveContainer" containerID="39ec76b0ad57a2d67853eaa4b682ba87609a6ef80562304999ccada56185f4e2" Sep 09 23:29:48.119259 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:29:48.118873 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:29:52.118691 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:52.118652 2580 scope.go:117] "RemoveContainer" containerID="55e737020be1c68908903f671bebbd067a5fe574dfd88d5a2a3eb8b9878e58cb" Sep 09 23:29:52.119193 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:29:52.118915 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:29:59.120884 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:29:59.120853 2580 scope.go:117] "RemoveContainer" containerID="39ec76b0ad57a2d67853eaa4b682ba87609a6ef80562304999ccada56185f4e2" Sep 09 23:29:59.121488 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:29:59.121109 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:30:03.118856 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:03.118819 2580 scope.go:117] "RemoveContainer" containerID="55e737020be1c68908903f671bebbd067a5fe574dfd88d5a2a3eb8b9878e58cb" Sep 09 23:30:03.990625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:03.990591 2580 generic.go:356] "Generic (PLEG): container finished" podID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" exitCode=0 Sep 09 23:30:03.990818 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:03.990663 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" event={"ID":"b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4","Type":"ContainerDied","Data":"cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a"} Sep 09 23:30:03.990818 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:03.990725 2580 scope.go:117] "RemoveContainer" containerID="55e737020be1c68908903f671bebbd067a5fe574dfd88d5a2a3eb8b9878e58cb" Sep 09 23:30:03.991083 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:03.991062 2580 scope.go:117] "RemoveContainer" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" Sep 09 23:30:03.991275 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:30:03.991251 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:30:07.601650 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:07.601606 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:30:07.602174 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:07.601680 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:30:07.602174 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:07.601698 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:30:07.602174 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:07.601712 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" Sep 09 23:30:07.602174 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:07.602128 2580 scope.go:117] "RemoveContainer" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" Sep 09 23:30:07.602345 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:30:07.602311 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:30:14.118950 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:14.118917 2580 scope.go:117] "RemoveContainer" containerID="39ec76b0ad57a2d67853eaa4b682ba87609a6ef80562304999ccada56185f4e2" Sep 09 23:30:14.119471 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:30:14.119150 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:30:22.118424 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:22.118385 2580 scope.go:117] "RemoveContainer" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" Sep 09 23:30:22.118939 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:30:22.118596 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:30:26.118978 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:26.118938 2580 scope.go:117] "RemoveContainer" containerID="39ec76b0ad57a2d67853eaa4b682ba87609a6ef80562304999ccada56185f4e2" Sep 09 23:30:26.119382 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:30:26.119162 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:30:37.118890 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:37.118854 2580 scope.go:117] "RemoveContainer" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" Sep 09 23:30:37.119454 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:30:37.119071 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:30:40.118854 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:40.118814 2580 scope.go:117] "RemoveContainer" containerID="39ec76b0ad57a2d67853eaa4b682ba87609a6ef80562304999ccada56185f4e2" Sep 09 23:30:41.216636 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:41.216548 2580 generic.go:356] "Generic (PLEG): container finished" podID="76b303ca-932f-4459-bd7d-3c1e70189b5e" containerID="b23e8e3f35672ec7d60344fed96544bc77fa6662b1a40499d8a6b712d4b6347f" exitCode=0 Sep 09 23:30:41.216636 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:41.216621 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" event={"ID":"76b303ca-932f-4459-bd7d-3c1e70189b5e","Type":"ContainerDied","Data":"b23e8e3f35672ec7d60344fed96544bc77fa6662b1a40499d8a6b712d4b6347f"} Sep 09 23:30:41.217184 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:41.216671 2580 scope.go:117] "RemoveContainer" containerID="39ec76b0ad57a2d67853eaa4b682ba87609a6ef80562304999ccada56185f4e2" Sep 09 23:30:41.217184 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:41.217090 2580 scope.go:117] "RemoveContainer" containerID="b23e8e3f35672ec7d60344fed96544bc77fa6662b1a40499d8a6b712d4b6347f" Sep 09 23:30:41.217341 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:30:41.217309 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:30:44.804692 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:44.804644 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:30:44.805314 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:44.804715 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:30:44.805314 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:44.804729 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:30:44.805314 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:44.804741 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" Sep 09 23:30:44.805314 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:44.805226 2580 scope.go:117] "RemoveContainer" containerID="b23e8e3f35672ec7d60344fed96544bc77fa6662b1a40499d8a6b712d4b6347f" Sep 09 23:30:44.805524 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:30:44.805475 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:30:48.118648 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:48.118614 2580 scope.go:117] "RemoveContainer" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" Sep 09 23:30:48.119080 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:30:48.118804 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:30:57.118908 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:30:57.118866 2580 scope.go:117] "RemoveContainer" containerID="b23e8e3f35672ec7d60344fed96544bc77fa6662b1a40499d8a6b712d4b6347f" Sep 09 23:30:57.119404 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:30:57.119120 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:31:02.119128 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:02.119094 2580 scope.go:117] "RemoveContainer" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" Sep 09 23:31:02.119523 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:31:02.119252 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:31:10.118801 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:10.118760 2580 scope.go:117] "RemoveContainer" containerID="b23e8e3f35672ec7d60344fed96544bc77fa6662b1a40499d8a6b712d4b6347f" Sep 09 23:31:10.119344 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:31:10.118929 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:31:17.125505 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:17.119475 2580 scope.go:117] "RemoveContainer" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" Sep 09 23:31:17.125505 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:31:17.119871 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:31:24.118191 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:24.118150 2580 scope.go:117] "RemoveContainer" containerID="b23e8e3f35672ec7d60344fed96544bc77fa6662b1a40499d8a6b712d4b6347f" Sep 09 23:31:24.118639 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:31:24.118346 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:31:32.119119 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:32.119081 2580 scope.go:117] "RemoveContainer" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" Sep 09 23:31:32.119533 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:31:32.119275 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:31:37.119204 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:37.119166 2580 scope.go:117] "RemoveContainer" containerID="b23e8e3f35672ec7d60344fed96544bc77fa6662b1a40499d8a6b712d4b6347f" Sep 09 23:31:37.119691 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:31:37.119373 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:31:47.118574 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:47.118539 2580 scope.go:117] "RemoveContainer" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" Sep 09 23:31:47.120874 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:31:47.118726 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:31:48.118485 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:48.118454 2580 scope.go:117] "RemoveContainer" containerID="b23e8e3f35672ec7d60344fed96544bc77fa6662b1a40499d8a6b712d4b6347f" Sep 09 23:31:48.118738 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:31:48.118622 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:31:52.723900 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:52.723851 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hd246_bbf2d253-13d6-483a-a536-cc7525f2b3e3/manager/0.log" Sep 09 23:31:53.085767 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:53.085742 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9nx9s_e42903f1-61a7-44b9-8062-8ee4e1b138b0/server/0.log" Sep 09 23:31:53.196983 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:53.196940 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nmx4g_f594896f-45b1-477d-8e1a-419edd1bc2c6/manager/0.log" Sep 09 23:31:53.423220 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:53.423146 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8c5c8967d-rwzxh_854fe1c7-968f-457c-b31d-caaa41a22153/copy-manifests/0.log" Sep 09 23:31:53.429208 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:53.429174 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8c5c8967d-rwzxh_854fe1c7-968f-457c-b31d-caaa41a22153/manager/0.log" Sep 09 23:31:55.359295 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:55.359267 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-tpxwn_1f55521b-1fe7-4069-b578-ec8b253486a5/limitador/0.log" Sep 09 23:31:55.467513 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:55.467484 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-r69dz_8d9d6c34-100e-433a-a8cc-5a4628d08de6/manager/0.log" Sep 09 23:31:55.807546 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:55.807520 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5_35ec7208-3462-40cc-8a7b-816a5d860510/istio-proxy/0.log" Sep 09 23:31:55.917276 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:55.917250 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zx8jf_c863aba1-e01d-4350-ae08-419d0a04e4d6/discovery/0.log" Sep 09 23:31:56.023731 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:56.023702 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bd8cf678b-bj5d8_6a250e59-3323-4852-9e47-4cc738212fa0/kube-auth-proxy/0.log" Sep 09 23:31:56.238190 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:56.238111 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-rv5wq_3f5a84b3-8aed-44ba-ab78-ddae43293248/istio-proxy/1.log" Sep 09 23:31:56.983962 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:56.983935 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-6685d86975-hkzrj_b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4/main/5.log" Sep 09 23:31:57.302463 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:57.302426 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdk667_ab05e351-2fbe-48ff-85fa-d34c892a86df/main/0.log" Sep 09 23:31:57.520934 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:57.520901 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_76b303ca-932f-4459-bd7d-3c1e70189b5e/main/5.log" Sep 09 23:31:58.118404 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:31:58.118375 2580 scope.go:117] "RemoveContainer" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" Sep 09 23:31:58.118793 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:31:58.118565 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:32:03.119177 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:03.119137 2580 scope.go:117] "RemoveContainer" containerID="b23e8e3f35672ec7d60344fed96544bc77fa6662b1a40499d8a6b712d4b6347f" Sep 09 23:32:03.119743 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:32:03.119336 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:32:05.877137 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:05.877108 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xxk7v_6b326930-e182-4769-b41a-f79a91a2a5db/global-pull-secret-syncer/0.log" Sep 09 23:32:05.946256 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:05.946225 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g4hc4_a8cfb746-8899-4809-b938-560d777b6c85/konnectivity-agent/0.log" Sep 09 23:32:06.012193 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:06.012161 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-56.ec2.internal_04642d3ff3a391e94be23b8faa39fe5f/haproxy/0.log" Sep 09 23:32:10.743122 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:10.743090 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-tpxwn_1f55521b-1fe7-4069-b578-ec8b253486a5/limitador/0.log" Sep 09 23:32:10.772376 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:10.772336 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-r69dz_8d9d6c34-100e-433a-a8cc-5a4628d08de6/manager/0.log" Sep 09 23:32:11.118741 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:11.118717 2580 scope.go:117] "RemoveContainer" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" Sep 09 23:32:11.118889 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:32:11.118872 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:32:12.196245 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:12.196220 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fd6fee9b-7f86-4020-9f6e-6964d50996b0/alertmanager/0.log" Sep 09 23:32:12.218416 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:12.218389 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fd6fee9b-7f86-4020-9f6e-6964d50996b0/config-reloader/0.log" Sep 09 23:32:12.241527 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:12.241508 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fd6fee9b-7f86-4020-9f6e-6964d50996b0/kube-rbac-proxy-web/0.log" Sep 09 23:32:12.262483 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:12.262462 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fd6fee9b-7f86-4020-9f6e-6964d50996b0/kube-rbac-proxy/0.log" Sep 09 23:32:12.291049 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:12.291029 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fd6fee9b-7f86-4020-9f6e-6964d50996b0/kube-rbac-proxy-metric/0.log" Sep 09 23:32:12.313619 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:12.313601 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fd6fee9b-7f86-4020-9f6e-6964d50996b0/prom-label-proxy/0.log" Sep 09 23:32:12.338315 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:12.338294 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fd6fee9b-7f86-4020-9f6e-6964d50996b0/init-config-reloader/0.log" Sep 09 23:32:12.461998 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:12.461911 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-56fd4848c7-5zb6j_522c3955-f498-45d8-85c9-852d52ff4efc/metrics-server/0.log" Sep 09 23:32:12.536912 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:12.536889 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bz42s_74dccb39-20df-48ef-9138-845b5264dd83/node-exporter/0.log" Sep 09 23:32:12.558235 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:12.558213 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bz42s_74dccb39-20df-48ef-9138-845b5264dd83/kube-rbac-proxy/0.log" Sep 09 23:32:12.576403 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:12.576381 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bz42s_74dccb39-20df-48ef-9138-845b5264dd83/init-textfile/0.log" Sep 09 23:32:13.049362 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:13.049335 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-pjndv_180bec08-a68e-4886-8f1f-126fafcafbc5/prometheus-operator/0.log" Sep 09 23:32:13.070469 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:13.070415 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-pjndv_180bec08-a68e-4886-8f1f-126fafcafbc5/kube-rbac-proxy/0.log" Sep 09 23:32:13.120785 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:13.120757 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5867bb85c5-rh944_ea6cf1f1-1b3d-429b-8328-c144abb4ce46/telemeter-client/0.log" Sep 09 23:32:13.145232 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:13.145204 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5867bb85c5-rh944_ea6cf1f1-1b3d-429b-8328-c144abb4ce46/reload/0.log" Sep 09 23:32:13.163911 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:13.163886 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5867bb85c5-rh944_ea6cf1f1-1b3d-429b-8328-c144abb4ce46/kube-rbac-proxy/0.log" Sep 09 23:32:15.119251 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:15.119217 2580 scope.go:117] "RemoveContainer" containerID="b23e8e3f35672ec7d60344fed96544bc77fa6662b1a40499d8a6b712d4b6347f" Sep 09 23:32:15.119640 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:32:15.119454 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:32:15.866123 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:15.866093 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f46cd58f7-ds2nf_546492f4-9265-41a1-9f87-9e07df18e82e/console/0.log" Sep 09 23:32:16.270162 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.270073 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q"] Sep 09 23:32:16.273516 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.273499 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.276097 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.276077 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pwd2c\"/\"kube-root-ca.crt\"" Sep 09 23:32:16.276238 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.276079 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pwd2c\"/\"openshift-service-ca.crt\"" Sep 09 23:32:16.277023 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.277005 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pwd2c\"/\"default-dockercfg-nkb97\"" Sep 09 23:32:16.284989 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.284940 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q"] Sep 09 23:32:16.332166 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.332127 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb2ee9db-f368-46c1-a644-450454cfd64e-sys\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.332166 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.332168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb2ee9db-f368-46c1-a644-450454cfd64e-lib-modules\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.332412 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.332221 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eb2ee9db-f368-46c1-a644-450454cfd64e-proc\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.332412 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.332257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eb2ee9db-f368-46c1-a644-450454cfd64e-podres\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.332412 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.332371 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdw4s\" (UniqueName: \"kubernetes.io/projected/eb2ee9db-f368-46c1-a644-450454cfd64e-kube-api-access-qdw4s\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.433854 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.433818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb2ee9db-f368-46c1-a644-450454cfd64e-sys\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.433854 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.433855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb2ee9db-f368-46c1-a644-450454cfd64e-lib-modules\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.434116 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.433882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eb2ee9db-f368-46c1-a644-450454cfd64e-proc\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.434116 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.433937 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eb2ee9db-f368-46c1-a644-450454cfd64e-proc\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.434116 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.433937 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb2ee9db-f368-46c1-a644-450454cfd64e-sys\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.434116 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.433995 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eb2ee9db-f368-46c1-a644-450454cfd64e-podres\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.434116 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.434036 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdw4s\" (UniqueName: \"kubernetes.io/projected/eb2ee9db-f368-46c1-a644-450454cfd64e-kube-api-access-qdw4s\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.434116 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.434039 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb2ee9db-f368-46c1-a644-450454cfd64e-lib-modules\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.434329 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.434138 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eb2ee9db-f368-46c1-a644-450454cfd64e-podres\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.441019 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.440998 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdw4s\" (UniqueName: \"kubernetes.io/projected/eb2ee9db-f368-46c1-a644-450454cfd64e-kube-api-access-qdw4s\") pod \"perf-node-gather-daemonset-cnc7q\" (UID: \"eb2ee9db-f368-46c1-a644-450454cfd64e\") " pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.585571 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.585539 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:16.853157 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:16.853132 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q"] Sep 09 23:32:16.854843 ip-10-0-130-56 kubenswrapper[2580]: W0909 23:32:16.854814 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podeb2ee9db_f368_46c1_a644_450454cfd64e.slice/crio-b8a1f104715fcda98d2838a75ec2507a6f7946e77545fc3d06b9af1669ce9eb9 WatchSource:0}: Error finding container b8a1f104715fcda98d2838a75ec2507a6f7946e77545fc3d06b9af1669ce9eb9: Status 404 returned error can't find the container with id b8a1f104715fcda98d2838a75ec2507a6f7946e77545fc3d06b9af1669ce9eb9 Sep 09 23:32:17.748741 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:17.748701 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" event={"ID":"eb2ee9db-f368-46c1-a644-450454cfd64e","Type":"ContainerStarted","Data":"4ac2c98dfb3181defb25c8db1becda9095cc874e6868573aef5fef59cde196c3"} Sep 09 23:32:17.749221 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:17.748770 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:17.749221 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:17.748795 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" event={"ID":"eb2ee9db-f368-46c1-a644-450454cfd64e","Type":"ContainerStarted","Data":"b8a1f104715fcda98d2838a75ec2507a6f7946e77545fc3d06b9af1669ce9eb9"} Sep 09 23:32:17.762878 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:17.762799 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" podStartSLOduration=1.762785192 podStartE2EDuration="1.762785192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:32:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:32:17.762297451 +0000 UTC m=+899.188720760" watchObservedRunningTime="2026-09-09 23:32:17.762785192 +0000 UTC m=+899.189208498" Sep 09 23:32:17.781908 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:17.781888 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5rjt2_5a33577f-f30a-4592-90e3-2915e9bad561/dns/0.log" Sep 09 23:32:17.799961 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:17.799938 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5rjt2_5a33577f-f30a-4592-90e3-2915e9bad561/kube-rbac-proxy/0.log" Sep 09 23:32:17.932642 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:17.932600 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fj28m_b22436d1-37ee-45c9-af11-3ee766cb8e70/dns-node-resolver/0.log" Sep 09 23:32:18.476550 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:18.476518 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7b6f97fb9d-wzhqp_567767c5-25a4-44b6-9f50-446d1ac6d3d6/registry/0.log" Sep 09 23:32:18.536890 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:18.536863 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v8n2v_887b5812-e7ca-4d43-a895-daa6e0005a8d/node-ca/0.log" Sep 09 23:32:19.172678 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:19.172653 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/0.log" Sep 09 23:32:19.174239 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:19.174220 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/0.log" Sep 09 23:32:19.413259 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:19.413226 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5_35ec7208-3462-40cc-8a7b-816a5d860510/istio-proxy/0.log" Sep 09 23:32:19.457659 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:19.457563 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zx8jf_c863aba1-e01d-4350-ae08-419d0a04e4d6/discovery/0.log" Sep 09 23:32:19.477725 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:19.477697 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bd8cf678b-bj5d8_6a250e59-3323-4852-9e47-4cc738212fa0/kube-auth-proxy/0.log" Sep 09 23:32:19.527797 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:19.527758 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-rv5wq_3f5a84b3-8aed-44ba-ab78-ddae43293248/istio-proxy/1.log" Sep 09 23:32:19.528168 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:19.528152 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-rv5wq_3f5a84b3-8aed-44ba-ab78-ddae43293248/istio-proxy/0.log" Sep 09 23:32:20.222668 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:20.222633 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kns46_58265cb5-d4cf-45b7-b827-7aa1717c810f/serve-healthcheck-canary/0.log" Sep 09 23:32:20.817778 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:20.817716 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ztk45_b464eccc-dbbb-4c62-9697-a23d8bdb846e/kube-rbac-proxy/0.log" Sep 09 23:32:20.837078 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:20.837052 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ztk45_b464eccc-dbbb-4c62-9697-a23d8bdb846e/exporter/0.log" Sep 09 23:32:20.856855 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:20.856828 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ztk45_b464eccc-dbbb-4c62-9697-a23d8bdb846e/extractor/0.log" Sep 09 23:32:22.970860 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:22.970816 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hd246_bbf2d253-13d6-483a-a536-cc7525f2b3e3/manager/0.log" Sep 09 23:32:23.066841 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:23.066811 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9nx9s_e42903f1-61a7-44b9-8062-8ee4e1b138b0/server/0.log" Sep 09 23:32:23.114123 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:23.114094 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nmx4g_f594896f-45b1-477d-8e1a-419edd1bc2c6/manager/0.log" Sep 09 23:32:23.225704 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:23.225632 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8c5c8967d-rwzxh_854fe1c7-968f-457c-b31d-caaa41a22153/manager/0.log" Sep 09 23:32:23.244123 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:23.244098 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8c5c8967d-rwzxh_854fe1c7-968f-457c-b31d-caaa41a22153/copy-manifests/0.log" Sep 09 23:32:23.764395 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:23.764371 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pwd2c/perf-node-gather-daemonset-cnc7q" Sep 09 23:32:24.431021 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:24.430994 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rlw7x_6025c052-7622-47b5-aaf7-902c23f42bd3/openshift-lws-operator/0.log" Sep 09 23:32:26.094330 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:26.094301 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5_35ec7208-3462-40cc-8a7b-816a5d860510/istio-proxy/0.log" Sep 09 23:32:26.118204 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:26.118175 2580 scope.go:117] "RemoveContainer" containerID="cefe7a20128a8cfc5ba52778a36192f2f3655c28110fa7db764f618b10bd727a" Sep 09 23:32:26.118415 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:32:26.118391 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-hkzrj_llm(b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-hkzrj" podUID="b4bc94ae-a9bb-439f-8fd3-8e64cc5a83d4" Sep 09 23:32:26.130463 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:26.130446 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zx8jf_c863aba1-e01d-4350-ae08-419d0a04e4d6/discovery/0.log" Sep 09 23:32:26.139432 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:26.139396 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bd8cf678b-bj5d8_6a250e59-3323-4852-9e47-4cc738212fa0/kube-auth-proxy/0.log" Sep 09 23:32:26.186490 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:26.186428 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-rv5wq_3f5a84b3-8aed-44ba-ab78-ddae43293248/istio-proxy/1.log" Sep 09 23:32:26.186848 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:26.186826 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-rv5wq_3f5a84b3-8aed-44ba-ab78-ddae43293248/istio-proxy/0.log" Sep 09 23:32:27.257026 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:27.256989 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nnnw5_35ec7208-3462-40cc-8a7b-816a5d860510/istio-proxy/0.log" Sep 09 23:32:27.294663 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:27.294639 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zx8jf_c863aba1-e01d-4350-ae08-419d0a04e4d6/discovery/0.log" Sep 09 23:32:27.306009 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:27.305956 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bd8cf678b-bj5d8_6a250e59-3323-4852-9e47-4cc738212fa0/kube-auth-proxy/0.log" Sep 09 23:32:27.344937 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:27.344912 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-rv5wq_3f5a84b3-8aed-44ba-ab78-ddae43293248/istio-proxy/1.log" Sep 09 23:32:27.345411 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:27.345398 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-rv5wq_3f5a84b3-8aed-44ba-ab78-ddae43293248/istio-proxy/0.log" Sep 09 23:32:30.118894 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:30.118862 2580 scope.go:117] "RemoveContainer" containerID="b23e8e3f35672ec7d60344fed96544bc77fa6662b1a40499d8a6b712d4b6347f" Sep 09 23:32:30.119285 ip-10-0-130-56 kubenswrapper[2580]: E0909 23:32:30.119073 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7_llm(76b303ca-932f-4459-bd7d-3c1e70189b5e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-ws2s7" podUID="76b303ca-932f-4459-bd7d-3c1e70189b5e" Sep 09 23:32:31.221060 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:31.221035 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbstm_410621c5-2a15-47eb-a4e4-842178ce68a7/kube-multus-additional-cni-plugins/0.log" Sep 09 23:32:31.239751 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:31.239729 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbstm_410621c5-2a15-47eb-a4e4-842178ce68a7/egress-router-binary-copy/0.log" Sep 09 23:32:31.260174 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:31.260153 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbstm_410621c5-2a15-47eb-a4e4-842178ce68a7/cni-plugins/0.log" Sep 09 23:32:31.282625 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:31.282605 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbstm_410621c5-2a15-47eb-a4e4-842178ce68a7/bond-cni-plugin/0.log" Sep 09 23:32:31.300914 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:31.300898 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbstm_410621c5-2a15-47eb-a4e4-842178ce68a7/routeoverride-cni/0.log" Sep 09 23:32:31.319557 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:31.319541 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbstm_410621c5-2a15-47eb-a4e4-842178ce68a7/whereabouts-cni-bincopy/0.log" Sep 09 23:32:31.337871 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:31.337856 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbstm_410621c5-2a15-47eb-a4e4-842178ce68a7/whereabouts-cni/0.log" Sep 09 23:32:31.397782 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:31.397744 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nw6cg_3c95a6dd-eb28-45c4-a536-b82de1e74698/kube-multus/0.log" Sep 09 23:32:31.484979 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:31.484905 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vdx58_63dd3bbf-4802-4b4a-9b06-37d9cf22ee94/network-metrics-daemon/0.log" Sep 09 23:32:31.501325 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:31.501302 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vdx58_63dd3bbf-4802-4b4a-9b06-37d9cf22ee94/kube-rbac-proxy/0.log" Sep 09 23:32:32.303863 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:32.303838 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-controller/0.log" Sep 09 23:32:32.321054 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:32.321031 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/1.log" Sep 09 23:32:32.321208 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:32.321142 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovn-acl-logging/0.log" Sep 09 23:32:32.337226 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:32.337198 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/kube-rbac-proxy-node/0.log" Sep 09 23:32:32.357754 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:32.357737 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/kube-rbac-proxy-ovn-metrics/0.log" Sep 09 23:32:32.375695 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:32.375675 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/northd/0.log" Sep 09 23:32:32.393939 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:32.393923 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/nbdb/0.log" Sep 09 23:32:32.412277 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:32.412253 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/sbdb/0.log" Sep 09 23:32:32.451732 ip-10-0-130-56 kubenswrapper[2580]: I0909 23:32:32.451712 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-66lv5_3132de00-554c-4204-b3f8-446e254f72e7/ovnkube-controller/0.log"